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I. INTRODUCTION 



Every year, thousands of new teachers pass through hundreds of different teacher 
preparation programs and are hired to teach in the nation’s schools. In recent years, “alternative 
teacher certification” (AC) programs or routes have expanded rapidly, offering an increasingly 
popular route into teaching that differs from that offered by traditional certification (TC) 
programs. TC programs generally require trainees to complete their training before they begin 
teaching full-time. In contrast, although AC programs vary with respect to their exact training 
strategies, they generally allow trainees to start teaching full-time before they have completed 
their training. 

Despite the expansion of these new routes into teaching, there exists little research to 
provide guidance as to the effectiveness of different teacher training strategies. The increased 
variation in teacher preparation approaches created by the existence of various AC and TC 
programs offers an opportunity to examine the effect of different components of teacher training 
on teacher performance. For example, some AC programs require substantially less education 
coursework than TC programs. We can exploit this type of variation to examine whether the 
form of teacher training is associated with differences in teacher performance. 

This document presents the design for an evaluation to investigate these issues. The 
evaluation — the Impact Evaluation of Teacher Preparation Models — is being sponsored by the 
U.S. Department of Education’s (ED’s) Institute of Education Sciences (lES) and is being 
conducted by Mathematica Policy Research (MPR), Decision Information Resources (DIR), 
Chesapeake Research Associates (CRA), and Vermont Institutes (VI). The purpose of the 
evaluation is to rigorously test whether the extent or content of teacher preparation is related to 
teacher practice or to the effectiveness of teachers, as measured by student academic 
achievement. 

Through various modes of data collection — both quantitative and qualitative — the study will 
determine the comparative effectiveness of contrasting methods of teacher preparation. To 
ensure a sufficiently large sample to detect meaningful effects, the study will follow two cohorts 
of students and teachers for an entire school year. The first cohort will be studied during the 
2004-2005 school year and the second cohort will be studied during the 2005-2006 school year. 
Students will be tested at the beginning and end of the school year to measure academic 
achievement. Teacher practices will be measured using classroom observations. Background 
information on teachers collected from a teacher survey and teachers’ SAT/ACT scores will be 
used to help interpret differences in student achievement and teacher practices among teachers 
who followed different preparation routes. 

The study will use a mix of experimental and nonexperimental methods to assess the 
relationship between preparation modes and student and teacher outcomes. Within each school 
in the study, students in the same grade will be randomly assigned to either a novice teacher from 
one of the AC models included in our study or a novice teacher from a TC route. This aspect of 
the design will provide an experimental estimate of the effect on student achievement of teachers 
from a particular AC model compared with teachers who choose a TC route. Using 
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nonexperimental analysis, we can then assess if that effect is related to the characteristics of 
preparation programs or to the characteristics of teachers who choose different programs. We 
will also compare the classroom practices of teachers who chose different preparation modes. 
The results of the study will be used to identify effective teacher preparation strategies and guide 
policy discussions on ways to promote the most effective among these strategies. 



A. QUESTIONS TO BE ADDRESSED BY EVALUATION 

The purpose of the impact evaluation is to determine the effect of different approaches to 
teacher preparation on teacher practice and student performance. Specifically, the evaluation 
will determine whether there is any difference in teacher practice or student achievement 
attributable to certain teacher preparation methods and, if so, which aspects of these routes are 
responsible for observed differences in teacher practice or student achievement. 

The evaluation will exploit the existing variation in teacher preparation in order to address 
questions in three specific areas: (1) professional preparation and support, (2) classroom 

practices, and (3) student performance. Questions that will be explored in each of these areas are 
as follows: 



• Professional Preparation and Support 

What is the nature and extent of teacher preparation in different preparation 
programs? What is the mix of training in such areas as pedagogy, classroom 
management, and child development, and how are these taught? What is the nature 
and extent of support for new teachers, provided either by the teacher preparation 
program or the school/district in which they begin their careers? 

• Classroom Practices 

How do different teacher preparation methods and different support and induction 
activities relate to new teachers’ classroom practices? 

• Student Performance 

Is the extent and content of teacher preparation, particularly the amount of education 
coursework, related to the effectiveness of teachers, as measured by student 
achievement? Is the timing of teacher preparation, specifically whether preparation is 
completed before assuming responsibility for a classroom, related to student 
achievements? 



The collection of information to address these questions will permit analyses that can inform 
the policy debate on appropriate strategies for responding to teacher shortages and promoting 
effective alternative preparation, traditional preparation, or aspects of both. 
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B. CONCEPTUAL FRAMEWORK FOR THE STUDY 



To understand the contribution of teacher preparation programs on teacher practices and 
student performance, it is important to account for differences in teachers’ personal and 
professional background characteristics, in addition to differences in the content of the 
preparation programs themselves. A conceptual framework for the study is depicted in Figure 
LI. 



This framework indicates core areas for exploration under the research questions posed in 
each of the three areas listed above. It also captures the key background characteristics that are 
hypothesized to influence the other measures of interest. The framework highlights the 
important links between the backgrounds of teachers — including such characteristics as age, 
education, academic ability, and previous work experience (column A); external factors resulting 
from the professional preparation received and possible ongoing support during their early years 
of teaching (column B); the intermediate effects these factors might have on classroom practices, 
which also are influenced by the factors associated with social context (column C); and the key 
longer-term effects on student performance, including achievement and school-related behaviors 
(column D). 



FIGURE I.l 

CONCEPTUAL ERAMEWORK FOR IMPACT EVALUATION OE TEACHER PREPARATION MODELS 
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Our evaluation is designed to examine how variations in teacher preparation programs, 
which are represented in column B of Figure LI, relate to different outcomes in classroom 
practices (column C) and student performance (column D). In the remainder of this report, we 
describe how we plan to conduct each component of the study, using Figure LI to guide our 
thinking. Our first step, which is described briefly in Chapter II, is to identify teacher 
preparation models that help us structure our thinking about variation in teacher preparation. 
The models differ both in terms of the characteristics of the teacher candidates they serve 
(column A) and the type of training and support provided (column B). Our objective in the 
study is to focus specifically on variations in training and support. The identification of the 
preparation models is based on examination of the existing alternative teacher preparation 
programs around the country. In Chapter II, once we have defined the preparation models, we 
will describe our approach for assessing the relationship between teacher preparation and 
classroom practices (column C) and our experimental design for assessing the effect of teacher 
preparation on student performance (column D). Chapter III addresses the plan for collecting 
data on all four parts of the conceptual framework shown in Figure 1.1. The data collection 
efforts will include: (1) administration of a teacher survey, (2) collection of teacher test scores, 
(3) observations of classroom instruction, (4) administration of standardized math and science 
tests to students, (5) collection of school records, and (6) collection of contextual information 
from programs and schools. Chapter IV describes how we will analyze all of these data to 
address each of the research questions. 
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II. EVALUATION DESIGN 



The study will address two broad research questions related to the impacts of teacher 
preparation programs on student outcomes and teacher practices: 



1. What are the relative impacts on student achievement of teachers who choose to be 
certified through different forms of teacher preparation? How do teacher practices 
vary by form of teacher preparation? 

2. How are student achievement and teacher practices associated with the timing and 
amount of teacher preparation coursework? 



The relative impacts on student achievement will be measured using an experimental design, 
where students in the study schools will be randomly assigned to classrooms taught by teachers 
from different preparation routes. Differences in teacher practice can be measured by comparing 
practices among teachers from different preparation routes. The remaining questions will be 
addressed using a mix of experimental and non-experimental research methods. 

This chapter describes our study design that will support the impact analysis. First, we 
discuss the types of teacher preparation models that will be the focus of the evaluation. Second, 
we provide an overview of the random assignment design. Third, we discuss the appropriate 
level of teaching experience for teachers in the sample. Finally, we present the sample design 
and the statistical precision of the impact estimates under this design. 



A. SELECTION OF TEACHER PREPARATION MODELS 

To structure our experimental design, we have chosen to study teacher preparation models 
that have substantive differences from one another in structure and amount of coursework, but 
that are also representative of the types of programs that train a majority of the nation’s teachers. 
In terms of structure, we focus on two types of programs: those that place teachers in classrooms 
only after all of their training has been completed (traditional programs) and those that place 
teachers in classrooms prior to the completion of their training (alternative programs.) Among 
the alternative programs, we further refine our focus based on the amount of coursework of the 
program. 

In previous research', we identified four prominent dimensions of variation that characterize 
alternative teacher preparation programs: (1) the entrance requirements, (2) the amount of 

required teacher-related coursework, (3) the institutions delivering the training, and (4) the 
amount of support provided during the first year of teaching. Although each of these dimensions 
merits attention, two stand out as especially important: the amount of teacher-related 

coursework and the entrance requirements. 



' Mayer, et al (2003). 
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The amount of education-related coursework required by AC and TC programs is critical to 
the debate over certification and teacher effectiveness. Some consider the required education 
coursework associated with traditional routes and some AC routes as unnecessarily burdensome 
(Finn 2003; Hess 2001; U.S. Department of Education 2002). These same critics argue that 
excessive coursework provides little benefit to those who take the courses and creates a 
disincentive for talented individuals to enter the teaching profession. On the other side of the 
argument, supporters of such coursework argue that reducing these course requirements will 
diminish the quality of the teaching force. Even in cases where AC coursework requirements are 
similar to TC requirements, the TC programs require that individuals complete these 
requirements prior to becoming the teacher of record, while AC programs allow teachers to 
become the teacher of record while they are still completing their requirements. The present 
study will inform the important debate regarding the importance of the amount and timing of the 
required education coursework in teacher preparation. 

The other dimension of variation across alternative certification programs on which we 
focus is the degree to which programs are selective in their recruitment of teacher candidates. 
Entrance requirements vary from less selective to very selective. Since entrance requirements 
might be closely related to teacher effectiveness, ignoring this fact would increase the odds that 
the study findings will confound the entrance requirements of a program with other aspects of a 
program. Hence, to identify the effects of different teacher preparation programs, we want to 
compare programs that serve broadly similar teacher candidates. 

Based on these two dimensions of variation, alternative certification programs can be 
divided into four categories according to entrance requirements (low or high selectivity) and 
course load (minimal or substantial). Table II. 1 illustrates these combinations of teacher 
preparation models. 

TABLE II. 1 

ALTERNATIVE TEACHER PREPARATION MODELS 





Entrance Requirements 


Highly Selective 


Less Selective 


Education 
Course Load 


Minimal 


Model A 


Model B 


Substantial 


Model C 


Model D 



Because of resource constraints, it will not be possible to include all models in the 
evaluation. Rather, we will select a subset of models, so that the evaluation can produce precise 
estimates of their effectiveness. Eor two reasons, the study will include only programs that have 
relatively low entrance requirements (models B and D). Eirst, most traditional certification 
programs do not have highly selective entrance requirements (Hess 2001), nor do most 
alternative programs (Mayer, et al 2003). Hence, programs with low selectivity are more policy- 
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relevant, since these are the programs that produce most teachers working today. Second, 
focusing on alternative certification programs with relatively low entrance requirements will help 
us disentangle effects of the teacher training from the effects of pre-existing individual 
characteristics. Finally, including both models B and D — programs with high course load 
requirements and programs with low course load requirements — will help us assess whether 
increasing the intensity of programs’ course load requirements improves teacher effectiveness. 



B. OVERVIEW OF THE RANDOM ASSIGNMENT DESIGN 

Our design will randomly assign students to practicing teachers (those who have primary 
responsibility for a classroom) who have chosen to enter the profession through traditional or 
alternative preparation routes. The key features of this design are to: 



• Select a geographically dispersed sample of novice teachers from two general types 
of alternative certification programs. The two types of programs that will be selected 
are those from programs with high or low course requirements (Models B and D). By 
“novice teachers,” we mean teachers with four or fewer years of experience as the 
teacher with primary responsibility for a classroom. 

• Identify the schools where these teachers teach. This information will be obtained 
from program directors of the selected alternative certification programs and the 
school districts served by the programs. 

• Identify traditionally certified teachers who teach in the same schools as the 
alternatively certified teachers. For each alternatively certified teacher in the sample, 
we will select a traditionally certified teacher who teaches in the same school and 
grade and who has the same amount of teaching experience (zero to four years of 
experience). 

• Randomly assign students to teachers within each school. Randomization will be 
conducted only in those grades taught by the teachers identified above. The random 
assignment of students to teachers will result in classes within schools that are 
essentially identical at the start of the school year. Thus, observed interclass 
differences in average student achievement at the end of the school year can be 
attributable to differences between the teachers and how they manage and instruct 
their classes. The random assignment of students to classrooms will avoid the 
possibility that students assigned to any given teacher will differ systematically from 
those assigned to any other teacher (which could lead to biased impact estimates). 
Random assignment also improves the precision of the impact estimates. 



^ An alternative approach is to randomly assign teachers to different types of programs. However, such an 
approach would fail to take into account the fact that trainees choose the program that best suits their needs and that 
trainees cannot generally be forced to attend a program they find unattractive. Since the characteristics of trainees 
and the programs they attend are inextricably connected in this way, this type of random assignment is clearly 
infeasible. 
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The goal of the random assignment design is to identify the relative effects of teachers who 
chose different teacher preparation models on student achievement. Our design will facilitate 
two types of comparisons using the models described above. First, we will compare teachers 
from Model B or D programs with other teachers in the same schools and grades who entered the 
profession through a traditional route. In the case of Model B, the comparisons will be between 
the traditional route teachers and alternative route teachers from programs with minimal 
coursework. The objective of this comparison is to measure the degree to which the differential 
coursework and timing of training of the teachers from traditional and alternative routes generate 
different student outcomes. In the case of Model D, the comparisons will be between traditional 
route teachers and alternative route teachers from programs with substantial coursework. Since 
coursework is substantial for both the alternative and traditional route teachers in this case, the 
objective of this comparison is to measure the degree to which the timing of training relative to 
placement in the classroom affects student achievement. These comparisons are made within 
schools with randomly created classrooms, so in effect we are conducting “mini-experiments” in 
each of the schools that will form the basis for our estimated impacts. 

Second, we will examine the relationship between student achievement and coursework 
requirements by comparing the outcomes of students taught by Model B teachers with those of 
students taught by Model D teachers. These two models will be compared using the traditionally 
certified teachers in each school as a common intermediary or benchmark. Specifically, the 
Model B impacts described above will be compared to the Model D impacts. Since both of those 
impacts are relative to the common benchmark of TC teachers, the two impacts can be compared 
to one another. This nonexperimental approach assumes that the TC teachers in schools with 
Model B teachers are, on average, comparable to the TC teachers in Model D schools. This 
assumption could be wrong if the types of schools that hire Model B teachers are systematically 
different from the types of schools that hire Model D teachers. We will use regression 
adjustment to control for observable differences between the TC teachers in the Model B schools 
and the TC teachers in the Model D schools. This comparison of Model B to Model D will show 
how coursework is related to student achievement for the teachers and AC programs in our 
sample, but may not be representative of Model B and Model D programs in general. 

One feature of our design that warrants comment is that because certification routes will not 
be randomly assigned to teacher trainees, the impact estimates under our design will address the 
effectiveness of alternate certification programs for those who choose to participate in these 
programs. It is, however, not structured to directly address the effectiveness of these programs 
for the average teacher, because of likely differences in the types of individuals who attend 
various certification programs. Stated another way, the results cannot be used to rigorously 
address how a graduate of one program would fare if they had instead attended another one. 
Instead, our design specifically addresses the effectiveness of teacher preparation programs for 
those they currently serve. 



^ Because we will collect extensive data on teacher characteristics, we can use non-experimental methods to 
partially disentangle impacts due to differences in training from those due to differences in teacher characteristics. 
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C. TEACHER EXPERIENCE 



We believe that the level of teaching experience should be similar for the alternatively and 
traditionally certified teachers. This is because we do not want to confound the comparison of 
alternative and traditional route teachers with different levels of teaching experience, especially 
since evidence suggests that teachers with at least a few years of experience, other things being 
equal, are more effective teachers than novice teachers."^ 

The study will focus on novice (instead of veteran) teachers, since the effect of teacher 
preparation is likely to be greatest for those with the least experience. For example, the 
structural difference between the traditional and alternative programs is that teachers in the 
alternative program are placed in a classroom prior to completion of their training. This initial 
gap in training between teachers from traditional and alternative programs may be closed over 
time as both gain classroom experience. Thus, the largest difference in student performance, if 
one exists, is likely to occur when teachers are new. Furthermore, different retention rates 
between teachers from different preparation routes could confound comparisons among veteran 
teachers. 

We will define “novice” to include teachers in their first four years of teaching. We include 
those with one to four years of prior experience, in addition to new teachers, because many first- 
year alternative route teachers are not yet certified. Thus, we believe the study sample should be 
designed so we can examine the performance of alternative route teachers both before and after 
they are certified. Furthermore, if the study were restricted to only new teachers, then it might 
not be possible to find enough schools with new teachers from both an alternative and traditional 
program in the same grade. If we are still not able to find enough teachers to meet our sample 
size requirements, we will expand the experience range to five years. 

In each school, teachers from alternative programs will be matched to teachers from 
traditional programs in the same grade and with a similar level of experience. Ideally, we would 
match novice teachers of exactly equal experience levels. However, we are aware that exact 
experience matches in the same school and in the same grade are too rare to meet the study’s 
sample size requirement. Given that, we will only require that all AC and TC teachers have at 
most four years of experience; we will not require that the experience levels match within each 
school. Using that approach, it is possible that either AC or TC teachers will have, on average, a 
bit more experience than the other group. In that event, we will use regression adjustment to 
account for differences in experience. 



D. SAMPLE DESIGN 

In this section, we describe our sample design, including our plans for selecting programs, 
school districts, schools, teachers, and students. Our design is a mixture of a “top-down” 
approach and a “bottom-up” approach. In the top-down approach, we start by selecting 
certification programs and work down to the schools who hired teachers from those programs. 



For example, Betts and Morell (1999) found that teacher experience in high school has a positive and 
statistically significant relationship with college GPA. 
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In the bottom-up approach, we look for additional alternatively certified teachers in the schools 
selected by the top-down approach, and then determine whether they were certified by programs 
that meet the requirements of this study. By including these additional teachers, we can bolster 
our sample in order to detect smaller effects. 

The sample design includes two cohorts of teachers corresponding to two study years. The 
first study year will begin with a smaller cohort (approximately one third of the ultimate sample), 
while the second study year will add a larger cohort of additional teachers. The second cohort 
will draw additional teachers from districts included in the first cohort as well as districts that 
will be new to the study. A small number of teachers who are in the first cohort will also be 
included in the second cohort. It should be feasible to identify a larger cohort of teachers in the 
second study year due to more time for recruiting and the benefit of lessons learned in the first 
year. 



1. Selection of Alternative Certification Programs 

As discussed above, we will sample 10 programs with low entrance requirements — 5 with 
minimal course requirements (Model B) and 5 with substantial course requirements (Model D). 
These 10 programs will be purposely selected from 165 programs in 12 states with low entrance 
selectivity requirements.^ To identify these 10 programs, we will first divide the universe of 165 
programs into seven strata based on geography and whether or not the program has intensive 
course requirements. Second, we will randomly select 50 programs across the 7 strata, where the 
sample size in a stratum will be proportional to the share of all programs that are in that stratum. 

Third, we will randomly order the selected programs within each stratum, and sequentially 
research each of the 50 programs to assess whether each one is appropriate for the study, based 
on certain selectivity requirements. These criteria will include (1) program size (the number of 
elementary school teachers trained by the program), so that we can select the larger programs to 
ensure that we will be able to obtain sufficient trainee samples; (2) program age, so that we can 
exclude programs that are too new (that is, programs that do not have teachers with 0, 1, or 2 
years of experience); (3) future operational plans, so that we can exclude those programs that are 
about to cease operations; and (4) entrance requirements, so that we can exclude programs that 
are too selective (that require a B average or better). This process will generate a sample of 10 
purposely selected but geographically dispersed programs that will be included in the study. 



2. Selection of School Districts 

We will request from the 10 programs identifying information on the districts where their 
program graduates have been placed in elementary schools in the previous three years. For each 
program, we will sample two school districts (we will select only one district if it is very large). 
To be eligible for inclusion in the study, a school district must have hired at least 12 teachers 
from the program within the past two years to ensure we will have a good chance of obtaining 

^ The 12 states are geographically diverse and include Arkansas, California, Colorado, Georgia, Illinois, 
Louisiana, Michigan, New Jersey, New Mexico, Tennessee, Texas, and Wisconsin. 
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our target sample of treatment and comparison group teachers. We will randomly select and 
order school districts from among those eligible and will sequentially contact them until our 
sample size targets have been met. 



3. Selection of Schools 

From each district, we will include about four schools in the study. Thus, the total number 
of schools in the study will be about 80. To be eligible for inclusion in the study, a school must 
have at least one alternatively certified teacher and one traditionally certified teacher with a 
similar amount of teaching experience (zero to four years) who teaches in the same grade. We 
will randomly sample and order schools in a district that meets these eligibility criteria, then 
sequentially contact these schools. 



4. Selection of Teachers 

At least one alternatively certified and one traditionally certified teacher with a similar 
amount of teaching experience will be included from each school in our sample. To increase 
precision, we will bolster this sample by including any other alternatively certified teachers 
present at a school, so long as their certification program meets our requirements, and so long as 
they have a traditionally certified match in the same grade and with similar experience. We 
assume that we will be able to sample a total of 90 alternatively certified and 90 traditionally 
certified teachers from the 80 schools in the two cohorts of the study, for a total of 180 teachers. 



5. Selection of Students 

We will request that school staff allow us to randomly assign all students in those grades to 
teachers included in the study. Thus, students in the relevant grades will be randomly assigned to 
a classroom taught by an alternatively certified teacher, a classroom taught by a traditionally 
certified teacher with comparable experience, or a teacher at the same grade level who is not 
included in the study (for example, a veteran teacher). This approach will reduce design effects 
due to clustering of students within classrooms because there should be no difference between 
the average baseline characteristics of students in the treatment and comparison group 
classrooms. All students in the treatment and comparison group classrooms will be part of the 
research sample. On average, we anticipate that 23 students will be in each type of classroom. 
Thus, we will have a sample of 80 schools, 90 alternatively certified teachers, 90 traditionally 
certified teachers, and 4,140 students. 



E. STATISTICAL PRECISION 

To assess appropriate sample sizes for the evaluation, we adopted a precision standard using 
impact results found in other evaluations. Several authors (for example, Cohen 1988; and Lipsey 
and Wilson 1993) have conducted meta-analyses across a range of fields to examine the extent to 
which impacts, measured in effect size units (that is, as a percentage of the standard deviation of 
the outcome), are considered to be “meaningful.” The consensus is that an effect size of .20 
(which is equivalent to about 4 percentile points on a standardized test) is considered to be 
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moderate in size. Thus, we adopt this .20 effect size value as the standard for this evaluation. 
Since subgroup analyses will be an important part of this evaluation, we have designed the study 
to achieve this standard for a 50 percent subgroup. 

Table II. 2 displays minimum detectable effects (MDE) on a student outcome measured in 
effect size units at 80 percent power for a 95 percent one-tailed test and for a 95 percent two- 
tailed test.^ The table also shows how the MDE varies with assumptions regarding the regression 
R value and subgroup size. These calculations incorporate design effects due to clustering at 
the school and teacher level. Incorporating design effects is necessary to calculate detectable 
effects that are externally valid to the programs included in the study. On the basis of findings 
from previous education-related evaluations, we assume an intraclass school effect of .07 and an 
intraclass teacher effect of .16. Other assumptions are displayed at the bottom of Table II2.. 

The expected student sample sizes provide sufficient statistical power to provide a definitive 
assessment of the overall impacts of the two types of teacher certification routes under 
investigation (relative to the traditional route), as well as for key subgroups of programs and 
teachers. Eor the overall design and a two-tailed test, the MDE is .13 of a standard deviation for 
an R value of .60 and .17 of a standard deviation for an R value of .30; these MDEs are both 
below our .20 precision standard. The MDEs are even smaller for a one-tailed test (.11 and .15, 
respectively). Eor a 50 percent subgroup of programs (for example, programs with substantial 
coursework requirements) or a 50 percent subgroup of teachers (for example, teachers in a 
particular range of grades), the MDE is about .18 of a standard deviation for an R^ value of .60 
and a two-tailed test or 0.24 for an R value of 0.30. Although we cannot know for sure what 
the R will be in these regressions, we believe this range suggests the study should be adequately 
powered to detect an effect size of approximately 0.20 of a standard deviation for a 50 percent 
subgroup under reasonable assumptions. 



'’The decision to use a one- or two-tailed test depends on the nature of the hypothesis being tested, which can 
vary both by outcome and by subgroup. Therefore, we present MDEs for both one- and two-tailed tests. 

^It is appropriate to assume an as high as 0.6 for outcomes where baseline measures of the outcome are 
available, such as test scores. However, there can be considerable variation in the regression from one study to 
another, thus an R2 of 0.6 should be considered a best case scenario. If baseline measures of the outcome are not 
available, we assume an R^ of 0.3. The considerable difference between the MDEs in these two cases (see Table 
II. 2) illustrates the importance of obtaining baseline measures of outcomes. 

* We can detect even smaller internally valid effects. When we do not take into account clustering, the MDE 
for a 33 percent subgroup is 0.10 standard deviations, assuming an R^ of 60 percent. Eor the full sample, the MDE 
is 0.06 standard deviations. 
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TABLE II.2 



MINIMUM DETECTABLE EEEECTS UNDER ALTERNATIVE SAMPLE DESIGNS 



Sample 




Detectable Effect Sizes (Percentage Points) 


Student 


(1) 


(2) 


Sample Size 
(Assuming 20 
per Class 
Complete 
Tests) 


Regression R^; 60% 


Regression R^; 30% 


1. One-Tailed Test. 80 Schools, 180 Teachers, 20 Students Responding per Teacher 


Pull Sample 


3,600 


11 


15 


50% Subgroup of Programs 


1,800 


16 


21 


50% Subgroup of Teachers 


1,800 


15 


20 


33% Subgroup of Programs 


1,200 


20 


26 


33% Subgroup of Teachers 


1,200 


19 


25 


25% Subgroup of Programs 


900 


23 


30 


25% Subgroup of Teachers 


900 


21 


28 


50% Teachers; 50% Programs 


900 


22 


29 


2. Two-Tailed Test. 80 Schools, 180 Teachers, 20 Students Responding per Teacher 


Pull Sample 


3,600 


13 


17 


50% Subgroup of Programs 


1,800 


18 


24 


50% Subgroup of Teachers 


1,800 


17 


23 


33% Subgroup of Programs 


1,200 


22 


30 


33% Subgroup of Teachers 


1,200 


21 


28 


25% Subgroup of Programs 


900 


26 


34 


25% Subgroup of Teachers 


900 


24 


32 


50% Teachers; 50% Programs 


900 


25 


33 



Note; Minimum detectable effects are estimated for a 5 percent level of significance and 80 percent power 
level. These calculations take into account clustering effects at the teacher level and at the school level. 
The equation used to calculate the minimum detectable effect is: 

2.486 . VUF . | 2a(1-0 ^2a.^1 (‘-A-A) 

\ S T N 

where 

S is the number of schools, T is the number of treatment (comparison) teachers, N is the number of 
students in the treatment (comparison) group, pi (=0.07) is the between-school variance as a percentage 
of the total variance of the outcomes based on previous studies, p2 (=0.16) is the between teacher 
variance, and c (=0.50) is the correlation between treatment and control group students within the same 
school. Previous impact evaluations have found that an R^ as high as 60 percent may be an appropriate 
assumption when baseline measures of an outcome are available, but 30 percent is more realistic when 
baseline measures are not available. However, the regression R^ can vary considerably between studies. 
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III. DATA COLLECTION PLAN 



Our data collection plan is designed to capture information that will permit analyses in the 
study’s three core areas of interest: (1) professional preparation and support of teachers, (2) 
classroom practices, and (3) student performance. Our goal is to apply data collection tools that 
will provide the most objective measures in each of these areas, that minimize respondent burden 
as much as possible, and that closely reflect the study’s needs as depicted in the conceptual 
framework (Figure LI in Chapter I). 

The primary data collection activities will include: 



• Administration of student tests in reading and math 

• Collection of teacher SAT/ACT test scores 

• Classroom observations 

• Teacher survey 

• Collection of students’ school records 

• Context study interviews 



These data collection activities will capture information called for in the conceptual 
framework presented in Chapter I as follows: The teacher survey, teacher test scores, and 
context study interviews will provide personal and professional background characteristics of the 
teachers (Column A of Figure II. 1) and information on teachers’ professional preparation and 
ongoing support (Column B); the classroom observations will provide an objective measure of 
classroom practice (Column C); and the student achievement test and school records data will 
provide the key outcome measures of student performance (Column D). 

The timeline for all data collection efforts is presented in Figure III.l. All data collection 
will take place during the 2004-2005 and 2005-2006 school years. We will collect baseline 
student achievement data at the start of each school year, during which time we will also request 
permission from teachers to access their SAT or ACT scores. Final student achievement data 
will be collected at the school year’s end. We will conduct classroom observations in the spring, 
and we will ask teachers to complete their survey at the time of the spring administration of the 
student achievement test. Context interviews with program and school staff will be conducted 
between January and March. We will collect school records data when available, following the 
end of each school year. 

Below we provide details on each of the six core data collection activities, with more 
detailed information on the objectives of each activity and our plans for collecting the 
information needed. The matrix presented in Table III.l displays the role of each data collection 
method in providing information relevant to the conceptual framework. 
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HGUREnU 



TIMELINE EOR DATA COLLECTION ACTTVITIES 









TABLE III. 1 



MATRIX OF DATA SOURCES AND DATA COLLECTION METHODS 



p 


Principal 


R 


School/District Records 


SAT/ACT 


Teachers' Scores on SAT/ACT tests 


ST 


Standardized Tests - MPR Administered 


T 


Teacher 


TC 


Traditional Certified Program Administrator 



AC 


Alt Cert Program Administrator 


D 


Certification Program Documents 


I 


Instructional Supervisor 


M 


Teacher Mentor 


CCD 


Common Core of Data (NCES) 







Data Collection Methods 




Topic Areas 


Phone Interview 


On-Site 


Survey 


Observation 


Document Review 




Personal Background of Teachers 




1 


Age 






T 






2 


Race/Ethnicity 






T 






3 


Gender 






T 






4 


Academic Ability 






T 




SAT/ACT 




Professional Background of Teachers 




5 


Education 






T 






6 


Nature, Extent of Work History 






T 






7 


Preparation to Teach 






T 






8 


Prior Classroom Experience 






T 






9 


Motivation to Teach 






T 






10 


Motivation to Select Route 






T 








Content of Preparation Program 




11 


Child Development 


TC 


AC 


T 




D 


12 


Classroom Management 


TC 


AC 


T 




D 


13 


Curriculum Content 


TC 


AC 


T 




D 


14 


Content-Specific Pedagogy 


TC 


AC 


T 




D 


15 


Diagnostics and Assessments 


TC 


AC 


T 




D 


16 


Instructional Logistics 


TC 


AC 


T 




D 


17 


Psychological and Moral Support 


TC 


AC 


T 




D 




Support Activities for Teachers 




18 


Courses 


TC 


AC, P/M/I 


T 




D 


19 


Mentoring 


TC 


AC, P/M/I 


T 




D 


20 


Observations 


TC 


AC, P/M/I 


T 




D 


21 


Personal Support 


TC 


AC, P/M/I 


T 




D 


22 


Other Induction Activities 




AC, P/M/I 


T 




D 


23 


Other Professional Activities 




AC, P/M/I 


T 




D 




Classroom Practices 




24 


Curriculum Coverage 








T 




25 


Pedagogical Practices 




P/M/I 


T 


T 




26 


Classroom Management 




P/M/I 


T 


T 






Teacher Commitment 




27 


Commitment to the profession 






T 






28 


Commitment to the school 






T 








Social Context 




29 


Community Expectations 




P 








30 


School Culture/Environment 




P 


T 




CCD 


31 


Family Characteristics 




P 








32 


Student Characteristics 










R 




Student Behavior 




33 


School Tardiness 










R 


34 


School Attendance 










R 


35 


Disciplinary Events 










R 


36 


Mobility (school, district, beyond) 










R 




Student Learning 




37 


Reading Achievement 










ST 


38 


Math Achievement 










ST 


39 


On-Time Promotion 










R 


40 


Recommended Summer School 










R 
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A. STUDENT TESTING 



Student achievement in reading and math is a critical outcome of interest in the study. Our 
intent here is to measure students’ math and reading achievement in a way that is both objective 
and consistent across the sample. Because student test scores will constitute the core outcome 
for teacher training impacts, it is critical that we are confident in their measurement and 
accuracy. Having trained MPR staff to administer the same test under the same testing 
conditions at approximately the same time of year to all students in the study sample will address 
these validity issues. 

We plan to administer four subtests of the California Achievement Tests, 5th Edition 
(CAT/5), published by CTB Macmillan/McGraw-Hill. The four subtests are: Reading 

Comprehension, Vocabulary, Math Concepts and Applications, and Math Computation. Each of 
these subtests has 20 questions and the total test time is 66 minutes. We will administer the same 
test form in the fall and in the spring. We chose to use the CAT/5 because it is a relatively short 
test and because the test generates scores for students who answer only a subset of questions, 
thereby avoiding lost observations due to incomplete tests. 

Prior to the test administration, we will need to secure parent/guardian permission for all 
sample members. We will distribute passive consent forms to all sample members at each school 
during the second week of classes; MPR staff will work with the schools to ensure that the forms 
are distributed. Schools will return to MPR via fax any form where the parents or guardians 
have indicated that their children cannot participate in the study. MPR staff will log the status of 
“no consent” into the data monitoring system. MPR staff will ensure that these students will not 
be tested. We expect less than 2 percent of the student sample will be excluded from the study 
because of non-consent.' 

The achievement tests will be administered in the fall approximately 4 to 5 weeks after the 
start of the school year, and in the spring approximately 19 to 20 weeks later. Prior to testing, 
MPR staff will arrange with each school for a convenient day and time to conduct the testing in 
each classroom. One week prior to the testing, MPR will reconfirm the testing schedule by 
sending a memo to each teacher. The memo will explain what the teacher can expect to do prior 
to our arrival as well as what we will want them to do during the testing period. 

The MPR testing team will consist of three staff: 1) the Test Administrator will give all 
verbal instructions to the students while they are taking the test; 2) the Test Proctor will walk 
around the classroom to ensure that all students are answering the test on the correct spot on the 
test form and ensure students are not looking at other classmates’ test materials to obtain 
answers; and 3) the Test Coordinator will ensure that all questions the principal and teacher may 
have are answered, and will help the Test Administrator and Proctor when needed. Before 
leaving the classroom, the MPR testing team will review all material to ensure students answered 
the questions. 



* These estimates are based on our experience with the Teach For America evaluation. 
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If for some reason a student did not complete the test (such as became ill in the middle of the 
test) they will be rescheduled for a make-up test session, along with any absent students. MPR 
local field staff will contact the schools to arrange a day to come in and administer the test to 
these students. We also expect that up to five percent of the student sample who were tested in 
the fall will have transferred to another school within the same original district by the time we 
test in the spring. Arrangements will be made by MPR staff with the principals of these new 

schools to test these students. Overall we expect 98 percent of the research students to complete 

2 

the test in the fall and 90 percent of the students to complete the test in the spring. 

Staff involved in test administration will participate in training approximately two weeks 
prior to fall testing. MPR staff will participate in training in person, while field staff will 
participate via telephone. Prior to training, field staff will be mailed a packet of training 
materials that will include an overview of the project and its purpose, all forms and surveys, 
procedures for each task, MPR policies, a schedule, and other documents. In the spring of the 
same school year, we will conduct a second training for any new staff who may be hired due to 
attrition. 

During the course of the study, some sample members will transfer to schools within 
district, and others will relocate outside the school district. Students who leave the district will 
not be tested again. For those who remain within district, we will attempt to test them in their 
new school during the same time period as those in their original cohort. We will track student 
mobility through the use of a mid-year classroom roster check in each school. Schools will be 
asked to send us their current rosters, as of January, and we will crosscheck these against our 
study sample in each classroom. Follow-up phone calls with the appropriate school 
administrator will help us determine the location of those students no longer enrolled in the study 
class. 



B. COLLECTION OF TEACHER TEST SCORES 

Teachers with varying levels of academic ability may demonstrate various levels of teaching 
effectiveness, regardless of their particular route to certification. As such, it will be important to 
control for differences in academic ability. Each sampled teacher will be asked for permission to 
allow the College Board or ACT to release their SAT or ACT scores, to us. These test scores 
will provide a basic measure of verbal and math ability. We will ask teachers for this consent at 
the time of the fall student achievement test administration when MPR staff will be on site. At 
this time we will address any questions or concerns the teachers may have about the test-score 
requests. It will be made clear to teachers that if they do not want us to have access to their 
scores, they may deny such access. 



^ These estimates are based on our experience with the Teach For America evaluation. 
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C. CLASSROOM OBSERVATIONS 



Measuring classroom practice will provide data on an important link between teacher 
preparation and student achievement. A key hypothesis of the study is that differences in teacher 
preparation will lead to differences in classroom practice that will ultimately affect student 
achievement. Conducting actual classroom observations of teachers in the study sample, rather 
than using self-reported data, will provide the most reliable source of information on a number of 
measures of classroom practice. The classroom observations will be conducted to gain first-hand 
knowledge of each study teacher’s approach to instruction in terms of their curriculum coverage, 
pedagogical practices, and classroom management. 

Our plan is to conduct four classroom observations for each teacher in the sample (both AC 
and TC teachers) during the 2004-2005 and 2005-2006 school years. Staff we hire and train 
would use a common tool to observe two reading units and two math units over the course of 
four days. The observers would observe all study classes within a district (anywhere from 10-18 
classes). Each observation will last 50 minutes, and will be preceded by a 10-minute interview 
with the teacher about the objectives for the planned lesson. 

We will use the Vermont Classroom Observation Tool (VCOT). This is a proprietary tool 
available only to individuals who receive training from the Vermont Institutes, which developed 
the instrument. The VCOT focuses on (1) planning and organization of a lesson; (2) 
implementation of a lesson; (3) content of a lesson; (4) and classroom culture in which the lesson 
is conducted. A series of items within each of these areas are then measured using a five-point 
rating scale. 

Vermont Institutes provides an intensive training, which is required before the tool may be 
used by anyone. The training is based on videotapes of actual classrooms, uses small groups to 
practice using the tool, and includes time to calibrate scores. 

MPR will supplement the VCOT training with our own process to ensure inter-rater 
reliability. Group leaders will accompany small groups of observers into schools to conduct pre- 
test classroom observations. All observers will score each protocol item independently. The 
group leader will facilitate a discussion of each item, spending time discussing items on which 
there is disagreement about the score. The group will come to a consensus score for each item, 
and the group leader will compute the percent agreement for each observer against the consensus 
score. To be certified to collect classroom observation data for the project, observers will be 
required to have rated items within one scale point of the consensus score on 80 percent of the 
items. 



D. TEACHER SURVEY 

The teacher survey is designed to measure many of the items included in the study’s 
conceptual framework. Topics to be covered are the personal and professional background of 
each teacher, the content of their teacher preparation program, the support activities they have 
received since becoming a teacher of record, their classroom practices, their commitment to the 
school and the profession, and their perceptions of the school culture and environment. In some 
cases it is our only source of information on things such as teacher background characteristics. 



20 




and in other cases it provides a critical perspective on issues such as professional preparation that 
will complement perspectives on similar topics covered in the context studies. 

We plan to administer in spring of each study year the teacher survey we developed (see 
Appendix A). Two weeks prior to the spring student achievement testing, we will mail a survey 
to each of the participating study teachers, along with a letter requesting the teacher to complete 
the survey prior to our arrival for spring testing. MPR staff will collect the survey on the day of 
the test. We expect that 75 percent of the teachers will complete the survey prior to MPR arrival. 
If the teacher has not completed the survey prior to our arrival, we will ask the teacher to 
complete the survey while MPR staff are administering the test to the students. We will also 
bring an extra copy of the survey for completion in the classroom, just in case the teacher left the 
survey at home. 

If the teacher is absent the day of testing, MPR field staff will follow up with the 
school/teacher to obtain the completed survey. We expect to achieve a 95 percent response rate 
for this survey. We expect five to seven percent of the classroom teachers present in the fall 
will have been replaced with a new teacher because of promotion, reassignment, illness or 
resignation. We will attempt to obtain contact information on the current location of these 
sample teachers and send a teacher survey to them for completion. In the cover letter to the 
exiting teacher we will explain the importance of their participation in the study and how they 
can return the completed survey to MPR using an enclosed postage -paid envelope. We will 
follow up by telephone, and email whenever possible. We expect to achieve an 80 percent 
response rate for this effort. 



E. SCHOOL RECORDS 

Collecting data from students’ school records provides additional outcome measures to 
supplement the achievement test scores. As depicted in the conceptual model, we plan to also 
measure student performance by such items as school attendance and disciplinary actions. 
Schools will be asked to provide records that contain data on each student’s demographic 
characteristics, school-related behaviors, and attendance and promotion. 

We will request school records data for each student in the summers of 2005 and 2006. We 
will mail records collection packages to schools at the end of each academic year. Prior to 
mailing the forms, we will inform the schools in a letter about their imminent arrival. The letter 
will also provide a toll-free telephone number school staff can use to call MPR to ask questions. 
We expect to obtain 90 percent of the records requested for students who remain within the 
districts in which schools are located. To reduce burden on schools, local field staff will be 
trained and available to help collect school records if a school chooses this option. MPR 
recognizes some schools may prefer to grant access to records solely to their own staff. 



^ These estimates are based on our experience with the Teach For America evaluation. 



21 




F. CONTEXT STUDY INTERVIEWS 



A final component of the data collection plan is the context studies of the teacher 
preparation programs and the schools included in the study. The objective of these context 
studies is to provide descriptive and qualitative data on the programs and schools, as well as to 
build up data measures related to program content and methods and school environmental 
factors, which can then be used in the data analysis. 

Program data will be collected primarily through semi-structured interviews of the directors 
of the teacher preparation programs. The protocols for these interviews are shown in Appendix 
B. Interviews with directors of all alternative certification program directors will be conducted 
in person, while interviews with the traditional program directors will be conducted either via 
telephone or in person. We expect that the in-person interviews will last approximately 60 
minutes and will yield data on the context of the preparation program and supplemental support 
provided by the program. As reflected in the protocols in Appendix C, the interviews will cover 
five areas in an attempt to differentiate among programs, including curriculum content, math and 
reading pedagogy, classroom management, student teaching, and mentoring. The questions are 
designed so that most responses can be coded quantitatively, using dichotomous, continuous, or 
scale measures. These coded data can then be included in the impact analysis, which is 
described in the next chapter. The interviews will be supplemented by the collection of 
documents, such as course lists or syllabi that provided relevant information about the programs. 

To collect contextual data from the schools, we will conduct semi- structured, in-person 
interviews with the school principals as well as teacher mentors, instruction supervisors, or other 
staff who are involved in providing support for new teachers. Again, we will also collect any 
relevant documents that provide contextual information on the schools and the support they 
provide to new teachers. The in-person interviews with will occur as part of site visits to school 
and programs between January and March. 



22 




IV. ANALYTIC APPROACH 



We will conduct two interrelated types of analyses to address the main impact-related 
research questions for the evaluation. First, we will compute impacts within each school (that is, 
conduct a series of “mini-experiments” in each school) by comparing the average outcomes of 
students taught by teachers who came through a particular alternative preparation model to those 
taught by traditionally trained teachers in the same schools and grades. Second, we will use the 
school-level impacts to compare the performance of teachers across the two alternative 
preparation models (programs with high and low course requirements). We will also examine 
whether impacts differ by teacher, school, and baseline student and family characteristics. 
Together, these analyses will provide policy-relevant information on the effects of different 
forms of teacher preparation on student outcomes (such as achievement and standardized test 
scores, attendance, promotion, suspensions, tardiness, and grades), and on teaching practices. 

Next, we discuss these analyses in more detail. The section begins, however, with a 
discussion of contextual analyses that we will conduct to aid in the interpretation of the impact 
estimates. Following that discussion is a description of the impact analysis, methods for 
addressing non-response, and a discussion of attrition and crossovers. 



A. CONTEXTUAL ANALYSES 

The evaluation will begin with several contextual analyses that will lay the foundation for 
the impact analysis, and will be crucial for interpreting the impact results. These analyses 
include: 



1. Assessing how well random assignment was implemented to examine the extent to 
which differences in outcomes between classrooms can be attributed to teachers. 

2. Examining the baseline characteristics of children in the classrooms in the study to 
understand the student population under investigation 

3. Examining the selectivity and curriculum requirements of the alternative and 
traditional certification programs included in the sample and the characteristics of 
teachers who attended different types of certification programs 



I. Assessing the Integrity of the Random Assignment Process 

The validity and interpretation of the impact estimates hinge on the integrity of the random 
assignment process and adherence to its procedures. We will conduct two analyses to gauge the 
success of the random assignment process. 

Eirst, we will examine the mobility of children in the sample into and out of treatment and 
control group classrooms using follow-up interview and school records data. Such movers 
complicate the analysis, because to preserve the integrity of the random assignment design. 
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children who relocate from treatment to control groups must be considered treatment group 
members in the analysis, and similarly, children who relocate in the reverse direction must be 
considered controls. As discussed later in this section, we will use statistical procedures to 
account for these crossovers. 

Second, within each school, we will conduct statistical tests to gauge the similarity of the 
baseline characteristics of children in the treatment and control classrooms. This analysis will be 
conducted using school records data. We expect that the random assignment used to select the 
classrooms in the research sample will produce equivalent treatment and control groups. 



2. Examining Sample Characteristics 

We will conduct descriptive analyses of the characteristics of the sample to help us more 
fully understand the types of children in the research sample, including their backgrounds and 
risk factors. These results will help us interpret program impact estimates, and indicate which 
subgroups are large enough to analyze. These analyses will be conducted using school records 
data. In addition, we will use published data to compare the characteristics of students in our 
sample to national samples of elementary school students. 



3. Examining the Characteristics of Certification Programs and Teachers 

A major component of the evaluation will be context studies of 80 schools and the relevant 
alternative and traditional certification programs of teachers included in the study. These context 
studies will be based on site visits to the schools and programs, including interviews with 
principals and program administrators. The study team will also conduct telephone interviews 
with administrators of the teacher preparation programs from which the traditionally certified 
teachers in the study originated. 

Data from these in-depth interviews, telephone interviews, and document reviews will be 
triangulated to produce a summary of information relating to the context of teachers certified 
through different routes of teacher preparation in each school sampled. A synthesis of these 
summaries will result in the development of cross-site comparisons that describe contextual 
similarities and differences for teachers certified through these different processes, as well as the 
nature and extent of their teacher preparation. 

An important part of this analysis will be to examine the characteristics of treatment and 
control group teachers using data from the teacher surveys. This analysis will be crucial for 
interpreting the impact estimates, because it will help us disentangle program effects due to 
differences in the backgrounds of treatment and comparison group teachers from program effects 
due to differences in the types of teacher certification training they received. 



B. IMPACT ANALYSES 

Our impact analyses are designed to gauge how different aspects of teacher preparation 
relate to student outcomes and teacher practices. First, we will estimate impacts (that is, average 
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treatment and comparison group differences) within each school, to find the effect of the 
structural differences between traditional and alternative programs. Second, we will compare 
impacts from these 90 mini-experiments across schools to find the effect of differences in the 
content of alternative programs (comparing model B to model D). Next, we describe these 
analyses in more detail, starting with a description of student and teacher outcomes. 



1. Student and Teacher Outcomes 

The study will examine the effect of different models of teacher preparation on student 
learning and teacher practices. These outcomes will be measured using a combination of school 
records, MPR-administered standardized tests, a survey of teachers, and classroom observations. 
Table IV. 1 summarizes the outcome measures and how they will be collected. 

The outcome of primary interest is student learning, which may be influenced by such 
mediating outcomes as teachers’ pedagogical practices, curriculum, and classroom management 
skills. Teachers who have completed their training prior to taking full responsibility for a 
classroom, and teachers from programs with a greater course requirement, may have better 
classroom management skills, may have a better command of the curriculum, and may have 
more effective teaching practices. All three of these factors could contribute to improved student 
learning. 

The outcomes described above are measured using data from a variety of sources. 
Standardized tests provide a measure of student achievement, while school records provide 
additional measures such as suspensions and expulsions. The teacher survey is designed to elicit 
accurate responses from teachers regarding their pedagogical practices, curriculum content, 
classroom management skills, and commitment to their profession. We have included multiple 
measures of pedagogical practices and curriculum content to capture important subtleties in these 
concepts. We will also explore using factor analysis to construct summary measures of these 
outcomes. Classroom observations will provide an additional perspective on teacher practices 
that can be used to balance the subjective views of teachers themselves. 



2. The Effect of Program Structure: Estimating Impacts from the Mini-Experiments 

Random assignment of students will be performed before the beginning of the school year. 
Thus, unbiased estimates of the impacts of being placed in a classroom taught by a teacher who 
chose either a type B or D alternative route program (relative to a teacher from the traditional 
route program) can be computed as the differences in the average outcomes of students in the 
treatment and comparison group classrooms. This approach yields unbiased estimates of the 
“intention-to-treat” impacts, because the random assignment design ensures that the primary 
difference between students in the treatment and comparison group classrooms at the point of 
random assignment to their teacher. Note, however, that this approach does not disentangle the 
characteristics of teachers who choose particular certification routes from the teacher training 
provided in each route, nor are the estimates necessarily representative of the universe of Model 
B and D programs. 
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TABLE IV.l 



OUTCOME MEASURES EOR THE TEACHER PREP EVALUATION, 
BY DOMAIN AND TYPE OE RESPONDENT 







Domain 




Construct 


School 

Records 


MPR- 

Administered Teacher 
Tests Survey 


Classroom 

Observation 


Student Performance and Classroom Environment 



Student Learning 

Reading Achievement Test Scores X 

Math Achievement Test Scores X 

On-Time Promotion X 

Recommended Attendance at Summer School X 



Student Behavior 



School Attendance Rates 


X 


Disciplinary Events 




Number of days suspended 


X 


Number of suspensions 


X 


Ever expelled 


X 



Teacher Classroom Management 

Classroom Culture^ 

Quality of the learning environment 
Level of student engagement 
Positive working relationships 
Sensitivity to diversity 

The Erequency with Which the Eollowing Has Occurred; 
Student tardiness 


X 


Student absenteeism 


X 


Physical conflict among students 


X 


Verbal conflict among students 


X 


Verbal abuse of teacher(s) 


X 


General misbehavior 


X 


Times per Week a Child Is Sent Out of Classroom for 
Misbehaving 


X 


Time Spent Each Day on the Eollowing Tasks: 
Academic instruction 


X 


Managing classroom behavior 


X 


Managing classroom tasks 


X 
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TABLE IV. 1 {continued) 







Domain 






MPR- 






School Administered 


Teacher Classroom 


Construct 


Records 


Tests 


Survey Observation 


Teacher Practices and Future Plans 


How Many Nights per Week Students Are Given Homework 
Average Time Students Expected to Spend on Homework 






X 


Each Night 






X 


Time Spent on Reading/Language Arts Instruction and 
Homework 








Hours per Week of English/Language Arts Instruction 






X 


Students Are Broken Into Reading Groups by Ability 






X 


Time Spent in Instructional Modes — Reading/Language 
Arts: 








Teacher-Directed Whole Class Activities 






X 


Teacher-Directed Small Group Activities 






X 


Students Working Independently in Small Groups 






X 


Students working individually on class assignments 






X 


Student-selected individual activities 






X 


Time Spent on Mathematics Instruction and Homework 








Hours per Week of Mathematics Instruction 






X 


Students Are Broken Into Math Groups by Ability 






X 


Time Spent in Instructional Modes — Mathematics: 








Teacher-Directed Whole Class Activities 






X 


Teacher-Directed Small Group Activities 






X 


Students Working Independently in Small Groups 






X 


Students Working Individually on Class Assignments 






X 


Student-Selected Individual Activities 






X 


Instructional Philosophies’’ 








Practices Phonics (Reading/Language Arts) 






X 


Practices Whole Language (Reading/Language Arts) 






X 


Practices Basic Skills (Mathematics) 






X 


Practices Application (Mathematics) 






X 
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TABLE IV. 1 {continued) 



Construct 


Domain 

MPR- 

School Administered Teacher 

Records Tests Survey 


Classroom 

Observation 


Qualitative Assessment Of Teacher Performance® 


Ability of Teacher to Plan/Organize a Lesson 




X 


Ability of Teacher to Implement Lesson 




X 


Quality of the Lesson’s Content 




X 


Commitment To Teaching 


Years Teacher Plans to Remain in Teaching 


X 




Future Career Plans 


X 





“These composite measures will be based on classroom observations using the Vermont Classroom Observation Tool. 

'’Instructional philosophies will be measured using composites based on teacher responses to statements regarding their 
approach to teaching language arts and math. 
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Although we will compute these simple differences-in-means impact estimates, we will 
focus on regression-adjusted estimates. Regression procedures will be used to improve the 
precision of the estimates and to adjust for residual differences in the observable characteristics 
of treatment and control group members, due to small sample sizes, random sampling, and 
interview non-response. We will estimate variants of the following regression model: 



(1) Y = Y,P,{SG^*T)+Xy + Y,d + e 

J 

where 

Y = Student outcome at a specific follow-up time point, such as standardized test 
scores 

SG = Indicator variable equal to 1 if the student is in school-grade combination j, and 0 
otherwise. 

T = Treatment indicator equal to 1 if the student is assigned to the treatment 
classroom (teachers from the selected alternative preparation model), and 0 if the 
student is assigned to the control group (teachers from a traditional preparation 
program) 

X = Student and family demographic characteristics pertaining to the period prior to 
random assignment, such as child’s gender, race/ethnicity, and family income 

Yo= Baseline measures of the outcome measures, such as student test scores 

Bj, y, S = coefficients to be estimated 

e = random (and mean zero) error component 



In words, equation (I) says that any given child outcome at a point in time is determined by 
the child’s baseline level of development, his or her family background, the intervention (in this 
case, the opportunity to be taught by a teacher from an alternative preparation program), and a 
set of other factors that are not related to his or her intervention assignment status. In this 
formulation, the estimate of represents the regression- adjusted impact estimate for school- 
grade combination j. Impact estimates across all schools and grades will be obtained by taking 
the average of the regression-adjusted impacts in each school- grade (that is, the y5js).^ The 
associated t-tests will be used to test the statistical significance of the impact estimates, where the 
standard errors will reflect design effects due to the clustering of students within classrooms and 
schools. 



* If the number of classrooms in each school varies substantially, then this technique will be modified to reflect 
that difference. 
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The explanatory variables included in the regression models will be obtained from the 
baseline tests conducted in the fall of each year, school records collected in the spring, and 
interviews with school and teacher preparation program officials. We expect that the 
explanatory variables will substantially increase the precision of the impact estimates, because 
some (in particular, the baseline test scores) are likely to be highly correlated with the outcomes 
measured at followup. The statistical methods used to estimate the regression models will 
depend on the nature of the outcome measure. For example, we will use ordinary least squares 
methods for continuous outcome measures (such as test scores), and logit maximum likelihood 
methods for binary ones (such as grade promotion). 

Findings from the regression adjusted impact analysis will be presented in easy-to-read 
tables. Table IV. 2 is a sample of the table shells that will be used to present impact findings. 
The table shows regression adjusted means for the treatment and control groups, the difference 
between the two, and the p-value corresponding to the statistical test of the hypothesis that the 
two means are equal. 



TABLE IV.2 

IMPACTS OF TEACHER CERTIFICATION ROUTE ON STUDENT OUTCOMES 



Treatment Control Estimated Effect 

Outcome Group Group Impact Size p-value “ 



Reading Achievement 

Math Achievement 

On-Time Promotion 

Recommended Attendance in Summer 
School 

Sample Size 



Source; School Records, MPR-administered standardized tests. 

Note: The treatment group consists of children taught by a teacher certified through a model B or D 

program (see Table II. I). The control group consists of children taught by a teacher certified 
through a traditional program in which teachers enter the classroom only after completing all 
training. 

“The p-value is the smallest level of significance at which the null hypothesis that the impact equals zero 
can be rejected. If the p-value is less than .01, an impact is significant at the 1 percent level; if the p-value is 
less than .05, the impact is significant at the 5 percent level, and so on. 

**Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 
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Finally, we will use methods very similar to those described above to examine the extent to 
which the teaching practices of teachers from different preparation routes differ. For these 
models, the dependent variables will be the specific teacher practices, defined using data from 
the teacher surveys and classroom observations. We will use both binary dependent variables 
that correspond to specific survey or classroom observation items as well as composite variables 
that we will construct based on multiple survey or observation items. Confirmatory factor 
analysis will be used to verify that the composite measures are appropriately constructed. Thus, 
the models will contain one observation per teacher, rather than one per student. 



3. The Relationship Between Program Content and Key Outcomes: Comparing Mini- 

Experiments Across Schools 

This non-experimental analysis will use regression adjustment to examine the relationship 
between the content of different teacher preparation programs and key outcomes. Specifically, 
this analysis will address the following research questions: (1) Are relative program impacts 
correlated with the coursework requirements of alternative teacher certification programs? (2) 
Are impacts correlated with key teacher characteristics and practices? (3) Are impacts correlated 
with students’ baseline characteristics? This analysis will also help us to disentangle the effects 
of specific aspects of certification programs from teacher characteristics. 

To address these questions, we will conduct analyses that attempt to explain the variation in 
the 90 impact estimates from the mini-experiments as a function of key program features, teacher 
characteristics, and student characteristics. These analyses will rely on regression adjustment 
and will be non-experimental in nature. 

Assessing Variation in Impacts for Subgroups Defined by Key Program Features and 
Teacher Characteristics. Impact results for the programs and teachers with different 
characteristics can provide important information on how to improve or expand teacher 
preparation programs. By disentangling the effects of teacher characteristics from program 
characteristics, the findings can help policymakers assess the types of teachers who become 
successful and which program characteristics are associated with successful outcomes. 
Specifically, by comparing the impacts for teachers from Model B programs to the impacts from 
Model D, we can infer the relationship between coursework requirements and student outcomes. 
By focusing on the impacts for teachers from Model D programs (relative to TC teachers), we 
can infer the relationship between the timing of course taking (either before or after certification) 
and student outcomes. It is important to note, however, that any correlation between impacts and 
program, teacher, or student characteristics is not necessarily causal. That is, the variation in 
impacts may be due to unobserved program, student, or teacher characteristics that happen to be 
correlated with the factors we do observe. 

Data used to define these subgroups will be obtained from the context studies (in-depth 
interviews during site visits, telephone interviews with program administrators, and document 
reviews), as well as the teacher surveys. We expect the program subgroups to include measures 
of the intensity and nature of the required coursework and the overall quality and cost of the 
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2 

program. The teacher subgroups will be defined by the characteristics of the AC teacher in each 
match and are likely to include teachers’ test scores, teaching experience, and employment 
history. The final set of subgroups will be made in consultation with IBS. The benefits of 
examining each of these subgroups are: 

Coursework. Analyzing subgroups of teachers defined by the coursework requirements of 
their certification program can inform the ongoing development of certification programs, by 
investigating whether there is a link between course requirements and student outcomes. 

Program Quality/Cost. A consistently applied definition of program quality, combined with 
a measure of the cost associated with higher quality, could help policymakers understand the 
trade-off between the benefits to students and the costs of a higher quality program. The 
definition of “quality” will be developed in consultation with IBS. 

Teacher Test Scores. There may be important interactions between teacher preparation 
routes and teachers’ own academic abilities. If the impacts of AC programs vary by teachers’ 
academic ability, then programs might reconsider their entrance requirements. 

Teaching Experience. We can create subgroups based on two types of teaching experience. 
Birst, we can define subgroups based on experiences where a teacher did not have primary 
responsibility for a classroom, such as student teaching or short term substitute teaching. 
Second, we can define subgroups based on experience as the person with primary responsibility 
for a classroom. Both analyses can help policy-makers understand the tradeoffs between 
teaching experience and other factors, such as coursework. However, both analyses are also 
subject to selection bias, since those who have greater experience may be systematically different 
in unobserved ways from those who do not. 

Employment History. AC teachers who have spent several years working in another 
occupation may differ from those with less work experience. Bor example, they might benefit 
from a greater overall level of maturity and professionalism. There are two implications of a 
differential effect of AC programs by employment history. Birst, employment history could be a 
factor to consider when admitting applicants to an AC program. Second, if we find that 
employment history is related to student outcomes, then we will need to take care to disentangle 
the effects of employment history from the effect of the AC program itself. As with teaching 
experience, estimates of the effect of employment history are also subject to selection bias, since 
those who choose to work in other occupations for an extended period of time could be 
inherently different from those who enter the teaching profession at a younger age. 

To start our analysis, we will estimate these subgroup relationships one at a time. We will 
group programs and teachers with a particular feature and estimate relationships separately for 
each group. Bor example, to estimate the impacts for programs with minimal course 



^ Using descriptive analyses, we will look for differences in the characteristics of the TC teachers between 
subgroups defined by AC teacher characteristics. If differences in TC teachers’ characteristics are systematically 
related to differences in AC teachers’ characteristics, we will attempt to regression adjust for the TC teachers’ 
characteristics. 



32 




requirements, we will estimate the impacts in those schools with treatment group teachers who 
were certified in programs with minimal course requirements. Similarly, to estimate the 
relationships between teacher preparation and outcomes for teachers that have two years of 
teaching experience, we will compare the outcomes of treatment and control group teachers with 
two years of teaching experience. We will estimate these relationships using the regression 
model in equation (1) and conduct statistical tests to gauge whether the relationships differ across 
levels of a subgroup (for example, across programs with intensive and minimal course 
requirements, or across teachers with different experience and age profiles). 

These relationships may be difficult to interpret if there is a high degree of correlation across 
key program features and teacher characteristics. For example, teachers with greater work 
experience may be concentrated in programs with a specific program model, making it difficult 
to disentangle the effects of each feature. Thus, we also will estimate hierarchical linear models 
(HLM) to simultaneously estimate the parameters associated with key program and teacher 
characteristics. These models will isolate the relationships between the outcomes and a 
particular program or teacher characteristic, holding constant the influence of other observable 
factors. These models cannot control for unobserved factors, however 

The HLM models will be estimated by regressing the impact estimates from each school 
(that is, the /3 in equation (1)), on the program and teacher measures (denoted by Z) as follows: 

(2) P = a + Zy + u, 

where a and y are parameters to be estimated and m is a mean zero term. The results from 
these models can be used to disentangle particular program or teacher features from others. 
Clearly, however, equation (2) can be estimated only if there is variation across the Z variables 
(that is, if the Z variables are not collinear). Thus, an important part of our initial analysis will be 
to conduct a detailed descriptive analysis to examine the correlation among potential Z variables 
and include only those variables that can yield meaningful results. 

We will also use an alternative, but complementary analytic approach to isolate the 
relationships between the outcomes and particular program features by estimating equation (1) 
for the full sample where the X variables include not only student characteristics but also 
measures of teacher characteristics. This approach will adjust for observable differences in the 
baseline characteristics of teachers who chose alternative certification routes, and hence, will 
attempt to isolate the effectiveness of programs’ training curricula on student outcomes. 

Findings from these subgroup analyses will be presented in easily interpretable tables. 
Table IV. 3 is a sample of the table shells that will be used to present subgroup findings. The first 
two columns show the estimated impacts for the subgroups of students who are taught by 
teachers from programs with substantial and minimum course requirements. The third column 
shows the difference in those impacts, which can be interpreted as reflecting the relationship 
between program course requirements and student outcomes. 
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TABLE IV.3 



IMPACTS ON STUDENT ACADEMIC OUTCOMES OE TEACHER TRAINING WITH 
MINIMAL AND SUBSTANTIAL COURSE REQUIREMENTS 





Estimated Impact 


Outcome 


Substantial Minimal Difference 

Coursework Coursework in Impacts Effect Size 


Reading Achievement 




Math Achievement 




On-Time Promotion 




Recommended Attendance in Summer 
School 




Sample Size 



Source: School Records, MPR-administered standardized tests. 

Note: Eor each estimated impact, the treatment group consists of children taught by a teacher 

certified through a model B or D program (see Table II.I). The control group consists of 
children taught by a teacher certified through a traditional program in which teachers enter 
the classroom only after completing all training. 

**SignificantIy different from zero at the .05 level, two-tailed test. 

***SignificantIy different from zero at the .01 level, two-tailed test. 



Assessing Variation in Impacts for Subgroups Defined by Student Characteristics. The 

relative effects of teacher preparation programs may vary across students with different 
characteristics because of the diversity in educational needs across students and the extent of 
teacher training required to effectively address these diverse needs. For example, it is possible 
that teachers from a given program may be less successful than teachers from a different 
program in promoting student achievement for students with low ability or for students in 
particular grades. Thus, understanding differences in program effects across student subgroups 
can help us determine what types of program models work best in different settings or for 
different types of teacher candidates. 

We will use the fall test scores and school records data to define the student subgroups. We 
expect that the subgroups will include grade, race and ethnicity, gender, and fall test scores, 
although the final list of subgroups will be selected in consultation with IBS. We will assess how 
the estimated impacts vary across student subgroups using procedures similar to those described 
above for the program- and teacher-related subgroups. Specifically, we will estimate equation 
(1) to compute regression-adjusted impacts for students in a particular subgroup. For example, 
we will estimate impacts for male students by comparing the mean outcomes of male students 
taught by treatment and control group teachers. In addition, we will include student subgroup 
indicator variables in the HLM models, to help disentangle the relationships between the 
outcomes and student, program, and teacher characteristics. 
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4. Accounting for Interview Non-response 



We expect high response rates to the interviews and assessments, because we expect that 
most children and teachers in the research samples will remain in the schools for the full year, 
and because we will track down and test any students who leave their schools during the year, as 
long as they remain in the same school district. Nonetheless, it will be important to test and 
correct for potential interview non-response bias. 

We will examine the extent of non-response bias by comparing the baseline characteristics 
of respondents and non-respondents using school records and baseline test score data. We will 
also compare the characteristics of respondents in the treatment and comparison groups. We will 
conduct statistical tests (t-tests and chi-squared tests) to gauge whether the differences in 
characteristics are statistically significant. 

We will account for potential non-response bias in several ways. First, as discussed, we will 
use regression models to adjust for differences in the observable baseline characteristics of 
respondents in the treatment and comparison groups. Second, we will construct non-response 
weights that weight respondents according to their similarity to non-respondents. The more 
similar a respondent is to non-respondents, the more heavily he/she will be weighted in our 
analyses. 

These weights will be constructed using baseline characteristics to predict response at 
followup. Specifically, we will run a logistic regression of follow-up response status on baseline 
variables. Using the parameter estimates from this regression, we will calculate the predicted 
probability of responding at followup for every member of the baseline sample. The inverse of 
these predicted probabilities will be the non-response weights. Thus, this propensity scoring 
procedure will yield large weights for those respondents with characteristics that are associated 
with low response rates. Similarly, the procedure will yield small weights for those respondents 
with characteristics that are associated with high response rates. A different non-response weight 
will be calculated for each follow-up instrument. That is, missing follow-up grades will be 
modeled separately from missing test scores. 

Finally, we will explore the sensitivity of our impact estimates to non-response by 
calculating impacts with and without the non-response weights. 



5. Accounting for Sample Attrition and Crossovers 

Our research design assumes that random assignment will be implemented with reasonably 
high fidelity to the intervention design, with most treatment group children remaining with the 
assigned treatment teacher and most control group children remaining with the control teacher. 
While impacts are straightforward to estimate under these assumptions, there are two sorting 
mechanisms that may confound the results. First, some students may fail to participate for the 
entire year in their assigned class; that is, of the students who are randomly assigned and were 
enrolled in the class at the time of the baseline test, some may leave the school at mid-year. The 
inclusion of these leavers in the analysis could lead to diluted impact estimates. The other 
sorting mechanism is that some students who are assigned to a treatment teacher may “cross 
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over” into a class with a control teacher, or vice versa. To preserve the integrity of the random 
assignment design, these crossovers will retain their original research status in the analysis. 

We will employ various statistical procedures to account for leavers and crossovers in the 
analysis, and compare the robustness of findings using these various approaches. First, we will 
estimate impacts using the full sample. This approach will yield uncontaminated impact 
estimates if, as expected, there are only a small number of leavers and crossovers in our sample. 
A second approach is to estimate impacts while excluding the leavers and crossovers in both the 
treatment and the comparison groups. This approach, however, will produce unbiased estimates 
only if the (observable) characteristics of the leavers and crossovers in the two research groups 
are similar (which will occur if mobility decisions are not influenced by differences in the quality 
of the treatment and comparison group teachers). Thus, we will compare the characteristics of 
leavers and crossovers (and stayers) in the two research groups to assess the appropriateness of 
this approach.^ 

Third, if attrition is large and appears to differ by research status, we will use propensity 
scoring methods to generate impact estimates that control for attrition. To implement these 
methods, we will estimate a logit model of whether a sample member is a stayer using baseline 
test score and school records data that will be available for both the movers and the stayers. The 
predicted probabilities (propensity scores) from these models will then be used to select 
members of the comparison group, with replacement, who are most similar to members of 
stayers in the treatment group. We will then estimate impacts by comparing the outcomes of 
treatment group stayers to those of their matched comparison group members. 



^The estimates under this approach will be calculated using sample weights to account for differences in the 
characteristics of movers and stayers. The weights will be constructed using propensity scoring methods and will be 
constructed so that the weighted characteristics of the stayers in the treatment and comparison groups will be similar 
to the characteristics of the full population. 
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APPENDIX A 

SELF-ADMINISTERED TEACHER SURVEY 




0MB No.: 1850-0794 
Expiration Date: 07/31/07 



BARCODE LABEL 



Study of T eacher 
Preparation Models 

The United States Department of Education wants to protect the privacy of individuais who participate in surveys. Your 
answers wiii be combined with other surveys, and no one wili know how you answered the questions. This survey is 
authorized by iaw (20 U.S.C. 1221e.1). You may skip questions you do not want to answer. We hope, however, that you wiii 
answer as many as you can. 

The questions on this form ask about your background, your current teaching experiences, and your pians for the future. For 
each item, piease mark oniy one answer, uniess instructions say to “CHECK ALL THAT APPLY.” Thank you very much for 
heiping us to iearn more about teacher preparation. 



CONFIDENTIALITY 

We want you to know that: 

1. We are asking you these questions in order to gather information about your preparation 
and experiences as a beginning teacher. 

2. You may skip any questions you do not wish to answer, however, we hope that you answer 
as many questions as you can. 

3. Your responses wiii be combined with those of other teachers, and the answers you give 
wiii never be identified as yours. 



Mathematica Policy Research, Inc. 
Princeton, NJ 

ajohnson@mathematica-mpr.com 

www.mathematica-mpr.com 

For questions, call 800-568-8535 



According to the Paperwork Reduction Act of 1995, no persons are required to respond to a coiiection of information uniess it dispiays a vaiid 
0MB controi number. The vaiid 0MB controi number for this information coiiection is 1850-0794. The time required to compiete this 
information coiiection is estimated to average 30 minutes per respondent, inciuding the time to review instructions, gather the data needed, 
and compiete and review the information coiiected. if you have any comments concerning the accuracy of the time estimate(s) or suggestions 
for improving this form, piease write to: U.S. Department of Education, Washington, DC 20202. if you have comments or concerns regarding 
the status of your individuai submission of this form, write directiy to: U.S. Department of Education, institute of Education Sciences, 
Washington, DC 20208. 




INSTRUCTIONS 



Thank you for taking the time to complete this survey. Your responses are very important to us. 



• When answering questions that require marking a box, please use an “X.” 

• Follow all “GO TO” instructions after marking a box. If no such instruction is provided, 
you should continue to the next question. 

• Either a pen or pencil may be used. 



DEFINITIONS FOR THIS SURVEY 



Teaching Certificate {also called Teaching Credential or License) - A document issued by a State 
Board of Education which permits the holder to be employed in the public school system as an educator. 
Teaching certificates may be obtained through either traditional or alternative educaWon programs. 

Traditional teacher certification program - An educational program in which a candidate completes the 
necessary initial study leading to an entry level teaching certificate prior to beginning employment as a 
teacher in a school. Higher education institutions deliver the training, as part of a bachelor’s or master’s 
degree program. 

Alternative teacher certification program - A program designed for individuals who already have a 
post-secondary degree. Minimal to no education courses or training are required prior to beginning 
employment in a school. Candidates often take courses and receive training while teaching. Training is 
delivered by higher education institutions, state agencies, or local school districts. Full certification is 
received one to three years after beginning the first teaching job. 

Student teaching {also called practice teaching) - A full day school-based experience for students 
enrolled in a post-secondary education institution that is supervised by both a certified experienced teacher 
and a university or college supervisor. It is a requirement of pre-service teachers who have completed the 
education coursework leading to a degree and are seeking certification or licensure to teach in a public 
school. 

Professional development activities - Educational activities in which teachers participate to enhance their 
career growth. Such activities may be formal or informal experiences in the form of inservice workshops or 
continuing education courses. They are designed for teachers to develop their knowledge in a variety of 
areas, such as in-depth study of a content area, teaching strategies, education standards, student 
assessment, applications of technology to instruction, or classroom management. 
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A. PROFESSIONAL BACKGROUND CHARACTERISTICS 



A1 . Please describe your postsecondary education by completing the chart below. 

Name of College or 
University 



a. Bachelor’s degree 

1 □ Yes > 

0 □ No 



b. Master’s degree 

1 □ Yes > 

0 □ No 



c. Other completed 

degree > 

(Please Specify) 



A2. Are you currently working toward an advanced degree (for example, Master’s or PhD) or additional credits? 



1 □ Yes 
0 □ No 



Deg ree : 

NAME OF COLLEGE OR UNIVERSITY: 



MAJOR FIELD OF STUDY: 



Year 


Name of College or 
University 


Major field of study 


Minor field of study 


I I I I I 


I I I I I 






I I I I I 
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A3. Since graduating from college , have you held a 
paying job other than your current teaching 
job? 

1 □ Yes 

oD No, this is my first job 

since coiiege — > GO TO A6 



The next few questions are about jobs 
you’ve held since graduating from college. 



A5. Again since graduating from college, tell us 
about the job you held prior to your current 
teaching position that LASTED THE LONGEST. 

DO NOT include a job that was an official part 
of your Teacher Preparation program (for 
instance, student teaching). 

0 □ I ALREADY REPORTED MY LONGEST 
JOB SINCE COLLEGE IN THE 
QUESTION ABOVE ^ GO TO A6 

a. What was your job titie? 



A4. Since graduating from college, please tell us 
about the most recent job you held PRIOR TO 
your current teaching position. 

DO NOT include a job that was an official part 
of your Teacher Preparation program (for 
instance, student teaching). 



b. What were your responsibiiities? What did you 
do in this job? (Please Be Specific) 



What was your job titie? 



b. What were your responsibiiities? What did you 
do in this job? (Please Be Specific) 



c. Did you work directiy with chiidren in this job? 



□ Yes 



0 □ No 



c. Did you work directiy with chiidren in this job? 



□ Yes 



0 □ No 



d. Was this job in the pubiic sector, the private 
sector, or the non-profit sector? 

MARK (X) ONE ANSWER ONLY 

1 □ Pubiic 

2 □ Private 

3 □ Non-profit 



d. Was this job in the pubiic sector, the private 
sector, or the non-profit sector? 

MARK (X) ONE ANSWER ONLY 

1 □ Pubiic 

2 □ Private 

3 □ Non-profit 



e. How many years did you work in this job? 

I I I NUMBER OF YEARS 

(Enter “0 1 ” if worked less than one year) 



e. How many years did you work in this job? 



NUMBER OF YEARS 

(Enter “0 1 ” if worked less than one year) 
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A6. Which of the following statements most 
accurately describes the type of teaching 
certificate/license/ credential that you currently 
hold? 

States vary in the types of certificates they 
issue. Piease seiect from the fist beiow 
the statement that BEST describes the 
certificate/iicense/ 
credentiai that you hoid. 

MARK (X) ONE ANSWER 

1 □ A regular or standard state certificate 

Year certified | | | | | 



2 □ A certificate that is issued to candidates 
after satisfying aii requirements except 
the compietion of a probationary teaching 
period 

Year certified | | | | | 



3 □ A certificate that is issued to candidates 
with the expectation that additionai 
requirements be compieted, such as 
passing a test or coursework 



4 □ An emergency certificate or waiver that 
is issued for a specified time period to 
persons with insufficient teacher 
preparation 



5 □ Other (Piease Describe) 




6 □ lam not certified 



A7. From the list below, select the areas in which 
you are certified, or are pursuing certification. 

MARK (X) ALL THAT APPLY 

1 □ Generai eiementary education 

2 □ Biiinguai education 

3 □ A specific subject area or areas 

(Piease Specify) 

4 □ Speciai education (Piease Specify) 

5 □ Other (Piease Specify) 

6 □ NOT CERTIFIED AND NOT PURSUING 

CERTIFICATIONS GOTOA13 
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A8. Which of the following statements best describes how you earned and/or are earning your teaching 
certificate? 



MARK (X) ONE ANSWER AND INDICATE 
NAME OF PROGRAM/SCHOOL 



1 □ In a traditional teacher certification program 

(see below for definition) as part of a bachelor’s 
degree — > GOTOA10 

2 □ In a traditional teacher certification program 

(see below for definition) as part of a “5th year” 
or master’s degree 

3 □ As part of an alternative teacher certification 

program /see below for definition). 

4 □ Other (Please Specify) 



Traditional teacher certification program - An educational program in which a candidate completes the necessary 
initial study leading to an entry level teaching certificate prior to beginning employment as a teacher in a school. 
Higher education institutions deliver the training, as part of a bachelor’s or master’s degree program. 

Alternative teacher certification program - A program designed for individuals who already have a post-secondary 
degree. Minimal to no education courses or training are required prior to beginning employment in a school. 
Candidates often take courses and receive training while teaching. Training is delivered by higher education 
institutions, state agencies, or local school districts. Full certification is received one to three years after beginning the 
first teaching job. 



Name of Program/School 
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A9. Why did you choose this particuiar route to becoming certified? 



MARK (X) ALL THAT APPLY 

1 □ a. As an undergraduate, I planned to be a teacher and so took all necessary courses to become 

certified. 

2 □ b. As an undergraduate, I planned to teach but did not want to take the necessary courses to 

become certified. 

3 □ c. As an undergraduate, I didn’t have plans to teach. 

4 □ d. As an undergraduate, I was Intent on pursuing a specific, non-teaching career. 

5 □ e. I pursued teaching as part of my master’s studies. 

6 □ f. I chose a route to becoming a certified teacher that required coursework and training that fits 

my schedule. 

7 □ g. I chose a route to becoming a certified teacher based on a program that was conveniently 

located. 

8 □ h. I chose a route to becoming a certified teacher based on financial considerations. 

9 □ I. I chose a route to becoming a certified teacher based on the requirements of the school, 

district, or state In which I wanted to teach. 

10 □ j. I chose a route to becoming a certified teacher that would allow me to become certified while 

working full-time. 

11 □ k. Other reason fP/ease Spec/Tyj 



A10. Are you currentiy pursuing state certification OR additionai advanced, professionai certification? 



1 □ Yes 
0 □ No - 



GO TO A13 



A1 1 . Have you compieted aii of your coursework? 

1 □ Yes — >• GOTO A13 

0 □ No 



A12. Piease indicate when you began the program, how much of your coursework is compieted, and when you 
expect to finish. 



Date you began the program 

(Month/Year) 


MARK (X) ONE ANSWER 


Expected date of compietion 

(Month/Year) 


Amount of Coursework Compieted 


Less than 
1/4 


1/4 to 3/4 


More than 
3/4 


I I |/| I I I I 

Month Year 


1 □ 2 □ 3 □ 


1 1 |/| 1 1 1 1 

Month Year 
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A13. Indicate the number of years you’ve worked in schools, INCLUDING THIS SCHOOL YEAR, in any of the 
following positions (either part-time or full-time). 



a. Certified teacher 

b. Emergency certified teacher . 

c. Teacher aide 

d. Long-term substitute teacher 

e. Substitute teacher 

f. Other (Please Specify) 



NUMBER OF YEARS WORKED 

(Enter “0 1 " if worked less than one year) 

(Enter “00” if you have never worked in this capacity) 


INDICATE NUMBER OF YEARS IN EACH POSITION 


In this school 


In a different school 































A14. Before beginning your first teaching position, did you complete any student/practice teaching (see page i for 
definition) that was part of your teacher preparation program? 

1 □ Yes 

0 □ No GO TO B1 

A15. How long did your practice teaching last? 

MARK (X) ONE ANSWER 

1 □ 5 weeks or iess 

2 □ 6-9 weeks 

sD 10 weeks or more 
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B. CONTENT OF YOUR TEACHER PREPARATION PROGRAM 



B1. In the grid below, indicate how much emphasis each of the topics received in coursework taken in your 
teacher preparation program. 



How much emphasis did the program 
coursework place on each topic? 

MARK (X) ONE FOR EACH 



No ^ ^ Strong 
emphasis ^ ^emphasis 



a. 


Using computers in classroom instruction 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


b. 


Planning daily lessons 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


c. 


Planning extended blocks of instruction 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


d. 


Assessing students formally (through tests, etc.) 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


e. 


Assessing students informally, through daily monitoring 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


f. 


Interacting with parents 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


g- 


Understanding child development 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


h. 


Teaching reading/language arts 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


i. 


Teaching math 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


j- 


Classroom management 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


k. 


Locating resources; setting up classrooms 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


I. 


Working with children with special needs 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 


m. 


Instructional strategies 


0 □ 


1 □ 


2 □ 


3D 


4 □ 


n. 


Providing you with moral/psychological support 


0 □ 


1 □ 


2 □ 


3 □ 


4 □ 



B2. Overall, how well did your teacher preparation program prepare you to begin your first year of teaching? 

0 □ It did not prepare me well at all 

1 □ It provided me with some preparation, but not in all areas or to the extent needed 

2 □ It provided me with most of the preparation I needed for teaching 

3 □ It provided me with all the preparation I needed for teaching 
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C. SUPPORT ACTIVITIES FOR TEACHERS 



C1 . Since becoming a fuii-time teacher, how many 

days, if any, have you spent in schooi- or district- 
supported professionai deveiopment activities 
(see page i for definition)"? 

NOTE: DO NOT INCLUDE pre-teaching experiences 
or training as part of your teacher preparation 
program or coursework. 

a. Number of days spent in first year of teaching 

0 □ 0 

1 □ 1-5 

2 □ 6-10 

3 □ 1 1 or more 

b. Number of days spent in second year of teaching 

0 □ 0 

1 □ 1-5 

2 □ 6-10 

3 □ 1 1 or more 

4 □ i’ve oniy taught for one year 

c. Number of days spent in third year of teaching 

0 □ 0 

1 □ 1-5 

2 □ 6-10 

3 □ 1 1 or more 

4 □ i’ve oniy taught for two years 



C2. Which of the foiiowing content areas were 
covered in the professionai deveiopment 
activities you reported in question Cl ? 

MARK (X) ALL THAT APPLY 

1 □ Standards (content and performance) 

in an area that you teach 

2 □ Methods of teaching/pedagogy 

3 □ Seiecting exempiary instructionai materiais 

4 □ Appiications of technoiogy to instruction 

5 □ Student assessment, such as methods 

of testing, evaiuation, performance 
assessment, etc. 

6 □ Student discipiine and ciassroom management 

?□ Study of a content area you teach: READING/ 
LANGUAGE ARTS 

8 □ Study of a content area you teach: MATH 

9 □ Other area #1 (Piease Specify) 



ioD Other area #2 (Piease Specify) 



11 □ Other area #3 (Piease Specify) 



C3. in your first year of teaching, did you work with a 
master or mentor teacher (or fieid supervisor)? 

1 □ Yes 

0 □ No GOTO C10 
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C4. Using the chart below, describe your interactions with the master or mentor teacher (or field supervisor) in 
your first year of teaching. 



a. The mentor observed your 

classroom teaching? 

b. You observed the mentor’s 

classroom teaching? 

c. You met formally with 

the mentor? 

d. You received written feedback from 

the mentor? 

e. You met informally with 

the mentor? 

f. You engaged in some other activity 

with a master/mentor? (Please 
Specify) 



C5. If you met formally with the mentor, what was the average length of your meetings? 

□ 15 minutes or less 

□ 15 to 30 minutes 

□ 30 to 60 minutes 

□ More than 60 minutes 

C6. The master or mentor teacher (or field supervisor) described in C4 above was: 

MARK (X) ONE ANSWER ONLY 

1 □ A requirement of your teacher preparation program 

2 □ Something your school/principal provided, but NOT related to your teacher preparation program 

3 □ Both 

4 □ I’m not sure if having the mentor was a program requirement or just provided by the school 

C7. Did you have a second master or mentor teacher (or field supervisor) when you were a first year teacher? 

1 □ Yes 

0 □ No GOTO CIO 





How often did each of the 
following occur . . . 


How helpful was it 






MARK (X) ONE FOR EACH ITEM 


MARK (X) ONE TO INDICATE 
HELPFULNESS 


Never 


One 

time 

only 


2-3 

times a 
term 


At least 
once a 
month 


At least 
once a 
week 


Not 
at all 


Slightly 


Somewhat 


Very 


oD 

1 


1 

1 □ 


2D 


3D 


A 

-□ 


oD 


iD 


2D 


3D 


i 

oD 

1 


r~ 

1 □ 


2D 


3D 


— 

4D 


oD 


iD 


2D 


3D 


i 

oD 

1 


r~ 

1 □ 


2D 


3D 


— 

4D 


oD 


iD 


2D 


3D 


i 

oD 

1 


I 

1 □ 


2D 


3D 


— 

4D 


oD 


in 


2D 


3D 


i 

oD 

1 


I 

1 □ 


2D 


3D 


— 

4D 


oD 


iD 


2D 


3D 


oD 


r~ 

1 □ 


2D 


3D 


— 

4D 


oD 


iD 


2D 


3D 
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C8. Using the chart below, describe your interactions with this second master or mentor teacher (or field 
supervisor) in your first year of teaching. 



a. The mentor observed your 

classroom teaching? 

b. You observed the mentor’s 

classroom teaching? 

c. You met formally with 

the mentor? 

d. You received written feedback from 

the mentor? 

e. You met informally with 

the mentor? 

f. You engaged in some other activity 

with a master/mentor? (Please 
Specify) 



C9. The master or mentor teacher (or field supervisor) described in C8 above was: 

MARK (X) ONE ANSWER ONLY 

1 □ A requirement of your teacher preparation program 

2 □ Something your school/principal provided, but NOT related to your teacher preparation program 

3 □ Both 

4 □ I’m not sure if having the mentor was a program requirement or just provided by the school 



C10. Using the chart below, please indicate whether you received the following kinds of support WHEN YOU WERE 
A FIRST YEAR TEACHER. 



a. Reduced teaching schedule 

b. Seminars or classes for beginning teachers 

c. Extra classroom assistance (e.g., teacher aide, team teaching) 

d. Regular supportive communication with your principal, other administrators, or 

department chair 

e. Opportunities to observe other teachers 



MARK (X) ONE 
IN EACH ROW 


1 □ Yes 


0 □ No 


1 □ Yes 


0 □ No 


1 □ Yes 


0 □ No 


1 □ Yes 


0 □ No 


1 □ Yes 


0 □ No 



How often did each of the 
following occur . . . 


How helpful was it . . . 


MARK (X) ONE FOR EACH ITEM 


MARK (X) ONE TO INDICATE 
HELPFULNESS 




One 


2-3 


At least 


At least 












time 


times a 


once a 


once a 


Not 








Never 


only 


term 


month 


week 


at all 


Slightly 


Somewhat 


Very 




1 






— 










oD 

1 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 


i 


I 






— 










oD 

1 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 


i 


I 






— 










oD 

1 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 




I 






— 










oD 

1 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 


i 


I 






— 










oD 

1 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 




I 






— 










oD 


1 □ 


2D 


3D 


4D 


oD 


1 □ 


2D 


3D 



Prepared by Mathematica Policy Research, Inc. 



10 



Now we’d like to ask some questions about this school year. 



Cl 1 . DURING THIS SCHOOL YEAR, have you had an assistant teacher, teacher’s aide, or tutor who assists you 
In providing academic Instruction to students In your class during the regular school day? 

Do not include student teachers, or Special Education or Resource Room teachers who provide remedial 
instruction to students. 

1 □ Yes 

0 □ No GOTO C13 

C12. For each subject listed below, please Indicate whether the assistant teacher , teacher’s aide , or tutor provides 
or provided instruction to students. If yes, approximately how many hours per week did this assistant 
typically work with students on that subject? Also, what percentage of the students receive instruction from 
this assistant? 



Subject 

a. Mathematics 

b. Reading/language arts 

c. Science 

d. Sociai studies 



C13. DURING THIS SCHOOL YEAR, has another teacher at your school (not an assistant or aide or special 

education teacher) provided instruction in mathematics, reading/language arts, science, or social studies 
to some or all of the students in your class? 

1 □ Yes 

oD No— >'GOTOC15 

Cl 4. For each subject listed below, please indicate whether another teacher provides or provided instruction to 
students in your class. If yes, approximately how many hours per week do students receive instruction from 
this other teacher ? Also, what percentage of the students receives instruction from this teacher? 



Subject 

a. Mathematics 

b. Reading/language arts 

c. Science 

d. Sociai studies 

CIS. Did you have a student teacher assigned to you at any time during THIS SCHOOL YEAR? 

1 □ Yes 
0 □ No 



Taught? 


Number of 
hours per week 


MARK (X) ONE IN EACH ROW 


Percent of students taught 


No 


Yes 


Less 
than 1/4 


1/4 to 1/2 


1/2 to 3/4 


More 
than 3/4 


oD 


1 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 



Taught? 


Number of 
hours per week 


MARK (X) ONE IN EACH ROW 


Percent of students taught 


No 


Yes 


Less 
than 1/4 


1/4 to 1/2 


1/2 to 3/4 


More 
than 3/4 


oD 


1 □-> 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 n-> 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 □-> 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 


oD 


1 □-> 


l_l_l 


1 □ 


2 □ 


3 □ 


4 □ 
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D. CLASSROOM PRACTICES 



D1 . How many nights per week do students in your ciass typicaiiy have homework? 

oD Zero nights per week — > GO TODS 

— iD One night per week 

2 □ Two nights per week 
— sD Three nights per week 
4 □ Four nights per week 

— 5 □ Five nights per week 



V 

D2. Estimate the amount of time, in minutes, an average student in your ciass is expected to spend doing 
HOMEWORK on each weeknight it is assigned. Piease report aM assigned activities. 

I I I I MINUTES PER TYPICAL WEEKNIGHT HOMEWORK IS ASSIGNED 



The next few questions ask about the instruction you provide in reading. 

D3. During a typicai WEEK, how many hours do you spend on READiNG/LANGUAGE ARTS instruction? 

I I I HOURS PER WEEK 

D4. in terms of your READiNG/LANGUAGE ARTS iNSTRUCTiON, do you use different groupings of students in 
your ciassroom to teach students who iearn at different rates? 

1 □ Yes 
0 □ No 



D5. How much time during a TYPiCAL SCHOOL WEEK do students in your ciass spend in the foiiowing 
readinq/ianquaqe arts activities? 





MARK (X) ONE FOR EACH ITEM 




No time 


Less than 
1 hour 


1-2 hours 


2-4 hours 


Over 4 
hours 


a. Teacher-directed whole class activities 


oD 


1 □ 


2 □ 


3D 


4 □ 


b. Teacher-directed small group activities 


0 □ 


1 □ 


2 □ 


3D 


4 □ 


c. Students working independently in 

pairs/teams/small groups 


oD 


1 □ 


2 □ 


3D 


4 □ 


d. Students working individually on class 

assignments 


oD 


1 □ 


2 □ 


3D 


4 □ 


e. Student-selected individual activities 


0 □ 


1 □ 


2 □ 


3D 


4 □ 
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D6. How often do students in this ciass do each of the foiiowing readinq/ianquaqe arts activities? 



NOTE: Some of the activities may not be appropriate for the grade you teach. If that is the case, mark “0” for Never. 







MARK (X) ONE ANSWER FOR EACH ACTIVITY 






Never 


Once a 
month or 
less 


2-3 times 
a month 


Once/ 
twice a 
week 


Nearly 
every day 


Daily 


a. 


Work on learning the names of the letters 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


b. 


Listen to you read stories where they see the 
print (e.g., Big Books) 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


c. 


Perform plays and skits 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


d. 


Read text with strong phonetic patterns 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


e. 


Retell stories 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


f. 


Do an activity or project related to a book or 
story 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


g- 


Read text with patterned or predictable text .. 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


h. 


Work in a reading workbook or on a 
worksheet 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


i. 


Compose or write stories or reports 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


j- 


Engage in peer tutoring 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


k. 


Read text with controlled vocabulary 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


I. 


Read thematic or literature-based text 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 


m. 


Publish their own writing 


oD 


1 □ 


2 □ 


3 □ 


4 □ 


sD 



The next few items ask about the instruction you provide in math. 

D7. During a typicai WEEK, how many hours do you spend on MATH instruction? 

I I I HOURS PER WEEK 

D8. in terms of your MATH iNSTRUCTiON, do you use different groupings of students in your ciassroom to teach 
students who iearn at different rates? 

1 □ Yes 
0 □ No 
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D9. How much time during a TYPiCAL SCHOOL WEEK do students in your ciass spend in the foiiowing number 
skiiis and mathematics activities? 





MARK (X) ONE FOR EACH ITEM 




No time 


Less than 
1 hour 


1-2 hours 


2-4 hours 


Over 4 
hours 


a. Teacher-directed whole class activities 


oD 


1 □ 


2 □ 


3D 


4 □ 


b. Teacher-directed small group activities 


0 □ 


1 □ 


2 □ 


3D 


4 □ 


c. Students working independently in 

pairs/teams/small groups 


oD 


1 □ 


2 □ 


3D 


4 □ 


d. Students working individually on class 

assignments 


oD 


1 □ 


2 □ 


3D 


4 □ 


e. Student-selected individual activities 


oD 


1 □ 


2 □ 


3D 


4 □ 



D10. How often do students in this ciass do each of the foiiowing number skiiis and mathematics activities? 



NOTE: Some of the activities may not be appropriate for the grade you teach. If that is the case, mark “0” for Never. 







MARK (X) ONE ANSWER FOR EACH ACTIVITY 






Never 


Once a 
month or 
less 


2-3 times 
a month 


Once/ 
twice a 
week 


Neariy 
every day 


Daiiy 


a. 


Play math-related games 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


b. 


Do math problems from their textbook 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


c. 


Work in mixed-achievement groups on math 
activities 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


d. 


Explain how a math problem is solved 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


e. 


Complete math problems on the chalkboard . 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


f. 


Work on math problems that reflect real-life 
situations 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


g- 


Work on learning rules and steps to solve 
problems 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


h. 


Do worksheets or workbook pages 
emphasizing routine practice or drill 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


i. 


Solve math problems in small groups or with 
a partner 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


j- 


Work on problems for which there are 
several appropriate methods or solutions 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


k. 


Work on problems for which there is a single 
right answer 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 


1. 


Work on memorizing math facts 


oD 


1 □ 


2 □ 


3D 


4 □ 


sD 
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D1 1 . To what extent, if any, has each of the following hindered student learning in your classroom since the start of 
the school year? 



MARK (X) ONE ANSWER FOR EACH 









To a 




To a 








slight 


To some 


great 






Not at all 


extent 


extent 


extent 


a. 


Student tardiness 


1 □ 


2 □ 


3 □ 


4 □ 


b. 


Student absenteeism/ciass cutting 


1 □ 


2 □ 


3 □ 


4 □ 


c. 


Physicai confiicts among students 


1 □ 


2 □ 


3 □ 


4 □ 


d. 


Verbai confiicts among students 


1 □ 


2 □ 


3 □ 


4 □ 


e. 


Verbai abuse of teacher(s) 


1 □ 


2 □ 


3 □ 


4 □ 


f. 


Generai misbehavior (e.g., students taiking in ciass, refusai to 
foiiow ciassroom ruies) 


1 □ 


2 □ 


3 □ 


4 □ 



D12. During a TYPICAL WEEK OF TEACHING, how many times do you have to send a child out of the classroom 
(i.e., to the principal’s or guidance counselor’s office) because of misbehavior or disruption? 

I I I TIMES 

D13. During a TYPICAL SCHOOL DAY, what percentage of your classroom time do you spend in the following 
tasks? 

NOTE: Total should equal 100 percent. 



Teacher Activity Percent of Time 

a. Academic instruction | | | % 

b. Managing ciassroom behavior | | | % 

c. Managing ciassroom tasks (e.g., handing out papers, 

transitions between activities, etc.) | | | % 



TOTAL = 100 PERCENT 
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E. TEACHING AS A CAREER 



E1 . Approximately how many years do you think you will remain in teaching? 

I will probably teach for . . . 

I I I more years 

E2. Which of the following statements best describes your plans? 

MARK (X) ONE ANSWER 

1 □ I plan to teach at least until I am eligible for retirement 

2 □ I will probably continue teaching unless something better comes along 

3 □ I plan to leave teaching as soon as I can 

4 □ I plan to pursue another education-related career at some point 

5 □ I plan to pursue another career outside of the field of education at some point 

6 □ lam undecided at this time 

7 □ Other (Please Specify) 



E3. How satisfied are you with EACH of the following aspects of the TEACHING PROFESSION? 







MARK (X) ONE FOR EACH ITEM 






Very 


Somewhat 


Somewhat 


Very 






dissatisfied 


dissatisfied 


satisfied 


satisfied 


a. 


Opportunities for professional advancement .... 


1 □ 


2 □ 


sD 


4 □ 


b. 


Salary and benefits 


1 □ 


2 □ 


3D 


4 □ 


c. 


Professional prestige 


1 □ 


2 □ 


3D 


4 □ 


d. 


Intellectual challenge 


1 □ 


2 □ 


3D 


4 □ 
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E4. How satisfied are you with EACH of the foiiowing aspects of teaching at THiS SCHOOL? 









MARK (X) ONE FOR EACH ITEM 








Very 

dissatisfied 


Somewhat 

dissatisfied 


Somewhat 

satisfied 


Very 

satisfied 


a. 


Recognition and support from administration 


1 □ 


2 □ 


sD 


4 □ 


b. 


Avaiiabiiity of resources and materiais/ 
equipment for your ciassroom 


1 □ 


2 □ 


sD 


4 □ 


c. 


Your infiuence over schooi poiicies and 
practices 


1 □ 


2 □ 


3 □ 


4 □ 


d. 


Autonomy or controi over your own ciassroom 


1 □ 


2 □ 


3 □ 


4 □ 


e. 


Student motivation to iearn 


1 □ 


2 □ 


3 □ 


4 □ 


f. 


Student discipiine and behavior 


1 □ 


2 □ 


3 □ 


4 □ 


g- 


Opportunities for professionai deveiopment 


1 □ 


2 □ 


3 □ 


4 □ 


h. 


The principai’s ieadership and vision 


1 □ 


2 □ 


3 □ 


4 □ 


i. 


Procedures for performance evaiuation 


1 □ 


2 □ 


3 □ 


4 □ 


j- 


Professionai caiiber of coiieagues 


1 □ 


2 □ 


3 □ 


4 □ 


k. 


Parentai invoivement in the schooi 


1 □ 


2 □ 


3 □ 


4 □ 



E5. During the first semester of the current schooi year, approximateiy how many days of teaching did you miss 
for any reason? 

I I I I NUMBER OF DAYS MISSED IN FIRST SEMESTER (SEPTEMBER TO DECEMBER) 
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F. PERSONAL BACKGROUND CHARACTERISTICS 



FI . In what year were you born? 



F6. How many of your children are . . . 



I_l_l_l_l year 



F2. What is your ethnic background? 

1 □ Hispanic or Latino 
0 □ Not Hispanic or Latino 



a. under the age of 5? 



b. between the ages of 5 and 1 1 ? 



c. between the ages of 12 and 18? 



d. over the age of 18? 



F7. How far in school did your parents go? 



F3. Check the box or boxes that describes your race 
or ethnicity. 

MARK (X) ONE OR MORE 

1 □ White 

2 □ Biack or African American 

3 □ Asian 

4 □ Native Hawaiian or Other Pacific Isiander 

5 □ American Indian or Aiaska Native 



F4. Are you: 

1 □ Maie? 

2 □ Femaie? 



F5. Do you have any children? Include birth, adopted, 
foster, or stepchildren. 

1 □ Yes 

0 □ No —> GO TO F7 



ANSWER FOR BOTH A AND B BELOW 





(MARK ONE) 


(MARK ONE) 




A 


B 




Father 


Mother 




(or male 


(or female 




guardian) 


guardian) 


Did not finish high schooi .... 


□ 


□ 


Graduated from high schooi 






or equivaient 


□ 


□ 


After graduating from high 
schooi, attended a 
vocationai schooi, a junior 
coiiege, a community 
coiiege, or another type of 
two-year schooi 


□ 


□ 


After graduating from high 
schooi, went to coiiege but 
did not compiete a four-year 
degree 


□ 


□ 


Graduated from coiiege 


□ 


□ 


Master’s degree or 






equivaient 


□ 


□ 


Ph.D., M.D., or other 
advanced professionai 
degree 


□ 


□ 


Don’t know 


□ 


□ 
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F8. We may wish to contact you for further information that wouid be heipfui to this study. Wouid you be wiiiing 
to speak to a member of the research team over the teiephone? 

1 □ Yes 

0 □ No 

V 

F9. Piease provide us with contact information in the spaces beiow. This information wiii remain compieteiy 
confidentiai. 

HOME PHONE NUMBER 

Area Code Number 

WORK PHONE NUMBER 

Area Code Number 

EMAIL ADDRESS 

SOCIAL SECURITY NUMBER 
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APPENDIX B 

CONTEXT STUDY PROTOCOLS 




AC PROGRAM DIRECTOR INTERVIEW GUIDE 



A. PROGRAM DESCRIPTION 

1. What were the admission requirements and process for students who entered this 
program in 199_/200_? For example, GPA, recommendation letters, interviews, 
timing, experience, etc.? 

2. Outline the elementary teacher training program for us as it existed for students who 
entered in 199_/200_. When did candidates enter? What were the required 
activities/courses before they become teacher of record? What about during first year 
of teaching? After first year of teaching? How long did program take to complete 
(e.g., years/months). [If they refer to units or credit hours, get an understanding of 
that metric. \ 

3. How many students were in this program, training to become basic elementary 
teachers (regular or bilingual), in the fall of 199_/200_? {Be sure to count all cohorts 
enrolled at one time; when a given cohort was new, some students were probably still 
enrolled from previous cohort.) 



B. PROGRAM STAFF 

During the academic year beginning in fall 199_/200_, 

1 . How many total instructors did your program have? 

2. How many field supervisors and/or mentors were on the staff, if any? {Overlap is 
okay — some class instructors may also serve as field supervisors/mentors — but probe to 
understand its extent.) 

3. What was the average class size {number of students per course)! 

4. What proportion of staff (instructors, field supervisors, mentors) were certified or 
previously certified as teachers? 

C. PROGRAM CONTENT 

During the academic year beginning in fall 199_/200_, 

1 . Did the program suggest or recommend that teacher candidates use any specific strategy 
or approach in teaching reading? (E.g., phonics, whole language, etc.) 

2. Did the program suggest or recommend that teacher candidates use any specific strategy 
or approach in teaching math? (E.g., drill and practice, real-life/applied?) 

B.3 AC PD Data Collection Guide 




D. PROGRAM DURATION AND AMOUNT OF INSTRUCTION/TRAINING 



For elementary teacher candidates who 
entered your program in 199_/200_1 how 


Number of hours provided... 


many hours of instruction/training were 
required overall and how were those hours 


(A) 


(B) 


(C) 


(D) 


distributed over time? {Complete row 1). 


Total 


Before 


During first 


After first 


And how many total hours were provided in 
each of the areas listed below? {Complete 
column A; their best estimates are fine; rows 
2-6 do not need to equal row 1.) 




becoming 
teacher of 
record 


year of 
teaching 


year of 
teaching 



1 . Overall, to complete the program 

2. Classroom management 

3. Reading/language arts-specific pedagogy 

4. Math-specific pedagogy 

5. Student assessment 

6. Child development 



I I I I I I I I 



I I I I 




7. How many hours, if any, were they required to spend in field work (e.g., doing observations)? 

8. How many hours, if any, would they have spent in self-paced or self-directed instruction {e.g., 
computer modules, as opposed to classes with an instructor present)! 



E. MENTORING AND OBSERVATIONS OF FIRST- YEAR TEACHERS 

1 . Did this program provide a mentor for. ..teacher name. . .during his/her first year of 

teaching? For future reference during this conversation, would you happen to know 
his/her name? If no mentor, skip to Section F. 

2. Can you briefly describe the support activities and services that the mentor from this 
program would have provided to. ..teacher name.. .during his/her first year as teacher 
of record, such as mentoring or being observed? (E.g., get specifics on number/length 
of observations, class meetings, solo meetings with mentor, etc.) 

F. OTHER ACTIVITIES 

1. For elementary teacher candidates who entered in 199_/200_, were there any other 
formal instructional or support activities that we have not already talked about? 
Please tell me about those. (Contact hours/course s/workshops, at what point in the 
training program, type of instructor, etc.?) 



B.4 



AC PD Data Collection Guide 



G. CLOSING (FINAL QUESTION ABOUT PROGRAM DURING TIME STUDY 
TEACHERS WERE IN IT) 



1 . Is there anything else I should know about the program requirements or how it 
operated? 



H. QUESTIONS ON CURRENT PROGRAM COSTS 
Substantive Questions 

(If respondent does not know answers, get referral to someone who would be able to answer.) 

For candidates who entered this AC program during the current academic year (2004-05), what 
would be the total cost for them to complete the program, including tuition, mandatory fees, 
books and supplies, and any other required expenses? 

(Look for full “sticker price, ” not out-of-pocket expected costs. If program takes more than one 
academic year to complete, make sure cost estimates cover full period.) 

Considering grants or scholarships or other financial arrangements (rebates? bonuses?) 
provided by this institution (not federal/state financial aid), how much of the full cost would 
typical participants really expect to pay? 



Do you know what the “real cost” of the program is to train a teacher — factoring in any subsidies 
the institution receives? (If not, OK!) 
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TC PROGRAM DIRECTOR INTERVIEW GUIDE (Revised) 



A. PROGRAM DESCRIPTION 

1. What were the admission requirements and process for students who entered this 
program in 199_/200_? For example, GPA, recommendation letters, interviews, 
timing, experience, etc.? 

2. Outline the elementary teacher training program for us as it existed for students who 
entered in 199_/200_. When did candidates enter? What were the required 
activities/courses before they become teacher of record? What about during first year 
of teaching? After first year of teaching? How long did program take to complete 
(e.g., years/months). [If they refer to units or credit hours, get an understanding of 
that metric. \ 

3. How many students were in this program, training to become basic elementary 
teachers (regular or bilingual), in the fall of 199_/200_? {Be sure to count all cohorts 
enrolled at one time; when a given cohort was new, some students were probably still 
enrolled from previous cohort.) 

B. PROGRAM STAFF 

During the academic year beginning in fall 199_/200_, 

1 . How many total instructors did your program have? 

2. How many field supervisors and/or mentors were on the staff, if any? {Overlap is 
okay — some class instructors may also serve as field supervisors/mentors — but probe to 
understand its extent.) 

3. What was the average class size {number of students per course)! 

4. What proportion of staff (instructors, field supervisors, mentors) were certified or 
previously certified as teachers? 

C. PROGRAM CONTENT 

During the academic year beginning in fall 199_/200_, 

1. Did the program suggest or recommend that teacher candidates use any specific strategy 
or approach in teaching reading? (E.g., phonics, whole language, etc.) 

2. Did the program suggest or recommend that teacher candidates use any specific strategy 
or approach in teaching math? (E.g., drill and practice, real-life/applied?) 
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D. PROGRAM DURATION AND AMOUNT OF INSTRUCTION/TRAINING 



For elementary teacher candidates who 
entered your program in 199_/200_, how 
many hours of instruction/training were 
required overall and in each of the areas listed 
helow? {Rows 2-6 do not need to equal row 1.) 


Total number of hours provided... 


1 . Overall, to complete the program 


1 1 1 1 


2. Classroom management 


1 1 1 1 


3. Reading/language arts-specific pedagogy 


1 1 1 1 


4. Math-specific pedagogy 


1 1 1 1 


5. Student assessment 


1 1 1 1 


6. Child development 


1 1 1 1 



7. How many hours, if any, were they required to spend in field work (e.g., doing observations) 
not including their student teaching assignment? 

8. How many hours, if any, would they have spent in self-paced or self-directed instruction (e.g., 
computer modules, as opposed to classes with an instructor present)! 

9. How many weeks long was student teaching? 

10. How many hours per day were they at their assigned school for student teaching? 

1 1 . When student teaching, how many full-length school days were they expected to be solely in 
charge of their classrooms? (Typically they build up to this level of responsibility at the end.) 



B.8 



TC PD Data Collection Guide 





E. STUDENT TEACHING AND OBSERVATIONS OF CANDIDATES 



Please provide the information requested in the following table. 



During the time when ...teacher name... was engaged in student 
teaching... 


(a) 

Number of times 


(b) 

Average length 
(in minutes) 


1. How many times would he/she have been observed in action in 
his/her classroom by his/her field supervisor? 


1 1 1 




2. How long would have been the average observation, including 
discussions immediately before or after? 




1 1 1 1 


3. How many times would. . .feac/zer name. . .have attended a required 
class/seminar while student teaching? 


1 1 1 




4. How long would have been the average class/seminar? 




1 1 1 1 


5. How many other times would he/she have met with his/her field 
supervisor besides those observations, and excluding any required 
classes/seminars for student teachers? 


1 1 1 




6. How long would have been the average meeting? 




1 1 1 1 



F. OTHER ACTIVITIES 

1. For elementary teacher candidates who entered in 199_/200_, were there any other 
formal instructional or support activities that we have not already talked about? 
Please tell me about those. {Contact hours/courses/workshops, at what point in the 
training program, type of instructor, etc. F) 



G. CLOSING (FINAL QUESTION ABOUT PROGRAM DURING TIME STUDY 
TEACHERS WERE IN IT) 

1. Is there anything else I should know about the teacher training program requirements 
or how it operated? 
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H. QUESTIONS ON CURRENT PROGRAM COSTS (for teacher training program, not 
for all costs associated with getting bachelor’s degree) 

Screening Questions: 

Does this institution offer a 5th-year or master’s program for people who already have a 
bachelor’s degree in some subject other than education, but who want to become elementary 
teachers? 

If yes, continue; If no, end now. 

Is it a program in which candidates take all their coursework and earn certification before 
beginning a regular teaching assignment? 

If yes, continue; If no, end now. 

Substantive Questions 

(If respondent does not know answers, get referral to someone who would be able to answer.) 

For candidates who entered this 5th-year or master’s program during the current academic year 
(2004-05), what would be the total cost for them to complete the program, including tuition, 
mandatory fees, books and supplies, and any other required expenses? 

(Look for full “sticker price, ” not out-of-pocket expected costs. If program takes more than one 
academic year to complete, make sure cost estimates cover full period.) 

Considering grants or scholarships or other financial arrangements (rebates? bonuses?) 
provided by this institution (not federal/state financial aid), how much of the full cost would 
typical participants really expect to pay? 

Do you know what the “real cost” of the program is to train a teacher — factoring in any subsidies 
the institution receives? (If not, OK!) 
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PRINCIPAL INTERVIEW GUIDE (Revised) 



A. SOURCES OF NEW TEACHERS 

1. How many classroom teachers are on your staff? 

2. How many classroom teachers are from alternative eertification programs (were 
teaching while still reeeiving training, before full certification)? 

3. What are your main sources of new teachers (e.g., which programs) and what is the 
recruitment and hiring process? 

4. What are the key faetors you like to eonsider in hiring new teaehers? {priority or 
importance?) 

5. Do you have preferences about the types of teaeher programs you hire from? 
{AC/TC, why?) 

6. Did you have a role in hiring any of the teaehers in our study? Whieh one(s)? \lf 
none, skip to section C] 

7. Did any other factors, besides those we just discussed, come into play when the 
teaeher(s) were hired here? {Probe on number, type of candidates to choose from, 
and level of need for new teachers at the time these two were hired.) 



B. SUPPORT FOR NEW TEACHERS 

Determine which study teachers, if any, principal knew/observed here during their first year of 
teaching; question is only relevant for them. {If none, skip to section D.) 

1 . Can you briefly describe the support activities and services that would have been 
provided to the study teaeher(s) during their first year teaching here, such as 
mentoring or observations? {Distinguish whether activities are part of requirements 
associated with certification program.) 
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C. PEDAGOGICAL PRACTICES AND CLASSROOM MANAGEMENT THIS YEAR 



Seek ratings from person with best knowledge of each teacher’s current performance; if not 
the principal, consider assistant principal, instructional supervisor, lead teacher, mentor, etc. 







Teacher name (AC) 






Teacher name (TC) 




Compared with other 
teachers in the school, how 






















MARK (X) ONE EOR EACH ROW 




MARK (X) ONE EOR EACH ROW 


well does — teacher 










Substan- 


Substan- 








Substan- 


name — do the following? 


Substan- 
tially below 








tially 

above 


tially 

below 








tially 

above 




average 




Average 




average 


average 




Average 




average 


Reading/Language Arts 






















1 . Accurately discern the 






















specific learning needs 
of individual students in 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


2. Use advance planning to 






















meet student-learning 
needs in 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


3. Lead instructional 






















activities during 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


4. Modify instruction 






















during a 

reading/language arts 

lesson when necessary to 
meet individual needs?.... 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


Math 






















5. Accurately discern the 






















specific learning needs 
of individual students in 

math? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


6. Use advance planning to 






















meet student-learning 
needs in math? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


7. Lead instructional 

activities during math? ... 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


8. Modify instruction 






















during a math lesson 
when necessary to meet 
individual needs? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 
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Teacher name (AC) 






Teacher name (TC) 




Compared with other 
teachers in the school, how 






















MARK (X) ONE EOR EACH ROW 




MARK (X) ONE FOR EACH ROW 


well does — teacher 










Substan- 


Substan- 








Substan- 


name — do the following? 


Substan- 
tially below 








tially 

above 


tially 

below 








tially 

above 




average 




Average 




average 


average 




Average 




average 


Classroom 

Management 






















9. Establish and enforce 






















classroom rules and 
procedures? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


10. Manage classroom time 
to keep students on-task? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 1 . Encourage desired 






















student behavior through 
praise, support, etc.? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


12. Engage students in 

learning? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


General 






















13. Utilize parents and 

school resources? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 
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C. PEDAGOGICAL PRACTICES AND CLASSROOM MANAGEMENT THIS YEAR 
(Continued, if more than 2 study teachers at this school) 

Seek ratings from person with best knowledge of each teacher’s current performance; if not 
the principal, consider assistant principal, instructional supervisor, lead teacher, mentor, etc. 



Compared with other 
teachers in the school, how 
well does — teacher 
name — do the following? 




Teacher name (AC) 






Teacher name (TC) 




MARK (X) ONE EOR EACH ROW 

Substan- 

Substan- tially 

tially below above 

average Average average 


Substan- 

tially 

below 

average 


MARK (X) ONE EOR EACH ROW 

Substan- 

tially 

above 

Average average 


Reading/Language Arts 






















1 . Accurately discern the 
specific learning needs 
of individual students in 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


2. Use advance planning to 
meet student-learning 
needs in 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


3. Lead instructional 
activities during 

reading/language arts?.. 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


4. Modify instruction 
during a 

reading/language arts 

lesson when necessary to 
meet individual needs?.... 


1 □ 


2D 


3 □ 


4D 


sD 


1 □ 


2D 


3 □ 


4D 


5 □ 


Math 






















5. Accurately discern the 
specific learning needs 
of individual students in 

math? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


6. Use advance planning to 
meet student-learning 
needs in math? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


7. Lead instructional 

activities during math? ... 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


8. Modify instruction 
during a math lesson 
when necessary to meet 
individual needs? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 
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Teacher name (AC) 






Teacher name (TC) 




Compared with other 
teachers in the school, how 






















MARK (X) ONE EOR EACH ROW 




MARK 


(X) ONE FOR EACH ROW 


well does — teacher 










Substan- 


Substan- 








Substan- 


name — do the following? 


Substan- 
tially below 








tially 

above 


tially 

below 








tially 

above 




average 




Average 




average 


average 




Average 




average 


Classroom 

Management 






















9. Establish and enforce 






















classroom rules and 
procedures? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


10. Manage classroom time 
to keep students on-task? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 1 . Encourage desired 






















student behavior through 
praise, support, etc.? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


12. Engage students in 

learning? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 


General 






















13. Utilize parents and 

school resources? 


1 □ 


2D 


3 □ 


4D 


5 □ 


1 □ 


2D 


3 □ 


4D 


5 □ 



D. READING/MATH INSTRUCTION AND OTHER INITIATIVES 

1. How do teachers in the study grades address instruction and assessment in reading 
and math? Does this school have any particular emphasis or initiatives in these areas? 
What, why, when, whose initiative (school, district, state), how relate to study 
teachers? 

2. Are there any other school or district or state initiatives, such as a major reform or 
restructuring, that may affect what the study teachers are doing in their classrooms? 



E. CLOSING (END OF QUESTIONS ABOUT CURRENT YEAR STUDY TEACHERS) 

1. Is there anything else I should know about. . . study teachers ’ names. . . or about school- 
based support for new teachers? 
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F. QUESTIONS ABOUT WIUUINGNESS TO PARTICIPATE NEXT YEAR 



Mention our goal of getting more matches this year and that each school that participates will 
get $5,000 (overall, not per match). If they ask about data collection/burden, it would be 
generally the same as this year, except that teachers would not need to fill out the same survey 
again in full. 

1 . Do you anticipate that the pair(s) of teachers included in the study this year will be teaching 
the same grade next year? If yes: Would you be willing to have us use the same lottery process 
and include those teachers' students in the study next year? (Note any 
concerns/questions/conditions.) 

2. Do you anticipate any pairings of relatively new AC and TC teachers (less than 4 years of 
experience) at this or any other grade levels next year? If so, would you consider participating in 
the study again next year with those other potential paired classrooms? (Note any 
concerns/questions/conditions.) 

(If possible bring back a copy of the school staff roster and forward it to lead recruiter. If you 
feel comfortable, use and annotate it during discusussion of possible matches.) 

(If the answer to either of the two main questions is yes or maybe, thank them and indicate that 
another study team member will be calling to talk further about participating in the study next 
year. Regardless of the specific answers, tell the lead recruiter for this district what you learned 
ASAP. (Do not wait until completing full write-up.) 
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